Comprehension Test

Comprehension Test

In Brief

A comprehension test will evaluate whether or not the customer understands the marketing message explaining the value proposition. This eliminates a possible false-negative bias on smoke tests where the customer indicates they do not want the value proposition when actually they do not understand it.

Helps Answer

  • Does the customer understand the value proposition?
  • How could we explain the value proposition better?


  • Quantitative
  • Qualitative
  • Value proposition
  • Smoke test


Time Commitment and Resources

  • For B2C, it can take 1-2 hours offline or 24 hours online.
  • For B2B, participant recruitment times can vary widely.
  • 10-20 participants.

How to

  • Write out value proposition in 1-3 sentences.
  • Show the value proposition to a participant for a few moments, then remove it.
  • Ask them to explain the value proposition in their own words from memory.

Interpreting Results

If the participant’s explanation is roughly comparable to our own, we count that as a positive result. If not, then it’s a negative. For this sort of test, we generally want a sample size of about 20 people and a positive conversion of about 80 percent.

The conversion has to be very high because regardless of what our value proposition is, people should understand it.

Take note: if many of the participants use identical language to explain the value proposition back, it should be considered as possible alternative marketing messages.

Potential Biases

  • Confirmation bias: Overly enthusiastic entrepreneurs will sometimes over-explain, correct, or nonverbally prompt the participant with the correct answer.
  • Invalid target audience: Participants do not need to be the target customers, but they must have the same level of language and vocabulary as the target customer (e.g., a junior marketing manager can stand in for a chief marketing officer).
  • False negative: When using online surveys such as FiveSecondTest, the distractions of an online test can often result in a higher than normal failure rate.

Field Tips

  • Run a comprehension test before a landing page test or you won’t understand why it doesn’t work. - @TriKro
  • Got a tip? Add a tweetable quote by emailing us: [email protected]

Case Studies



© All Rights Reserved            updated 2024-06-15 15:55:43

results matching ""

    No results matching ""