Generative vs. Evaluative

Is our question open-ended with many possible answers or does our it have a single true answer?

Similar to market vs. product, we can separate our questions into those that require generating ideas vs. those than provide factual answers.

Generative

• Who is our customer?
• What are their pains?
• What job needs to be done?
• How can we solve this problem?
• How are they doing this job today?
• Is our customer segment too broad?
• How do we find our customers?
• How much will this customer segment pay?
• How do we convince this customer segment to buy?

Evaluative

• Will this specific customer segment pay $9.99 for a solution?
• Should we target segment A or segment B?
• What is the cost of acquiring a customer in this customer segment?
• Will this specific feature increase our sales?

Telling them apart

The easiest way to tell if you have a generative question is whether or not it is open-ended. If the answer to the question could be a list of possible ideas, it’s generative.

If the answer to the question is closed-ended, it’s probably evaluative. Yes/no, multiple choice, questions about a specific fact are strong candidates. Quantitative questions in general are likely evaluative questions where there is one clear factual answer calculated from the data.

However, in some cases a question can appear to be closed-ended but have an ambiguous, poorly defined hypothesis backing it. To that end, the next chapter will provide some best practices around defining hypotheses.

Where Should We Start?

As with market vs. product, this book is agnostic about where we start. However, unless we can tie a clear and well defined hypothesis to an evaluative question, we are almost always better off starting with generative methods.