Real Startup Book
A comprehension test will evaluate whether or not the customer understands the marketing message explaining the value proposition.
Blog Posts & Articles
A comprehension test will evaluate whether or not the customer understands the marketing message explaining the value proposition. This eliminates a possible false-negative bias on smoke tests where the customer indicates they do not want the value proposition when actually they do not understand it.
- Does the customer understand the value proposition?
- How could we explain the value proposition better?
- Value proposition
- Smoke test
Time Commitment and Resources
- For B2C, it can take 1-2 hours offline or 24 hours online.
- For B2B, participant recruitment times can vary widely.
- 10-20 participants.
- Write out value proposition in 1-3 sentences.
- Show the value proposition to a participant for a few moments, then remove it.
- Ask them to explain the value proposition in their own words from memory.
If the participant’s explanation is roughly comparable to our own, we count that as a positive result. If not, then it’s a negative. For this sort of test, we generally want a sample size of about 20 people and a positive conversion of about 80 percent.
The conversion has to be very high because regardless of what our value proposition is, people should understand it.
Take note: if many of the participants use identical language to explain the value proposition back, it should be considered as possible alternative marketing messages.
- Confirmation bias: Overly enthusiastic entrepreneurs will sometimes over-explain, correct, or nonverbally prompt the participant with the correct answer.
- Invalid target audience: Participants do not need to be the target customers, but they must have the same level of language and vocabulary as the target customer (e.g., a junior marketing manager can stand in for a chief marketing officer).
- False negative: When using online surveys such as FiveSecondTest, the distractions of an online test can often result in a higher than normal failure rate.
- Run a comprehension test before a landing page test or you won’t understand why it doesn’t work. - @TriKro
- Got a tip? Add a tweetable quote by emailing us: [email protected]