Set your target
Lean Experiment Design

The Real Startup Book

A large, crowdsourced compilation of inquiry methods (i.e. experiment and research techniques) from lean practitioners worldwide. Every method has a short description, how to guide, and links to additional information.

Set your target
 

Problem vs. Solution

Do we need to learn about the problem (ie, market) or the solution (ie, product)?
Blog Posts & Articles
Validation
Innovation Methodologies

Hypothesis Checklist

A sure way to keep our sanity is to start with a strong hypothesis to give the experiment structure.
Blog Posts & Articles
Validation
Innovation Methodologies

Generative vs. Evaluative

Do we have a clear hypothesis to evaluate or do we need to generate a clear idea?
Blog Posts & Articles
Validation
Innovation Methodologies

Tags and Other Frameworks

What Are You Trying to Learn?

“If I had an hour to solve a problem, I'd spend 55 minutes thinking about the problem and 5 minutes thinking about solutions.” — Albert Einstein
Blog Posts & Articles
Validation
Innovation Methodologies

Customer Discovery Interviews

Gain insight into customers' perspective, pain points, purchasing habits, and so forth.
Blog Posts & Articles
Validation
Innovation Methodologies

Analog/Digital

This technique helps decouple the risks around physical delivery of a product from a digital component of the product.
Blog Posts & Articles
Validation
Innovation Methodologies

Focus Groups

A small group discussion designed to rapidly gain customer feedback on a given topic.
Blog Posts & Articles
Validation
Innovation Methodologies

Open-Ended Survey

Fixed set of questions, usually via a mass-communication channel such as an email or website pop-up.
Blog Posts & Articles
Validation
Innovation Methodologies

Comprehension Test

A comprehension test will evaluate whether or not the customer understands the marketing message explaining the value proposition.
Blog Posts & Articles
Validation
Innovation Methodologies

Secondary Market Research

Secondary market research gathers and interprets available information about the target market, and includes published reports, newspaper articles, or academic journals.
Blog Posts & Articles
Validation
Innovation Methodologies

Value Proposition Test - Broken Promise

This smoke test helps test how much of a natural/inherent virality lies in the idea itself. It helps identify high-potential ideas before we've built a product.
Blog Posts & Articles
Validation
Innovation Methodologies

Ask an Expert

Founders ask experts for opinions, then choose whether or not to act on those opinions.
Blog Posts & Articles
Validation
Innovation Methodologies

Value Proposition Test - Video

The video smoke test consists of using a trailer video to test excitement and virality/buzz when describing the product to an early-adopter audience.
Blog Posts & Articles
Validation
Innovation Methodologies

Closed-Ended Survey

Converge on what's relevant in great detail, particularly to customers or prospects.
Blog Posts & Articles
Validation
Innovation Methodologies

Competitor Analysis

Competitive analysis is very much a secondary research method that we can perform online relatively quickly and comprehensively.
Blog Posts & Articles
Validation
Innovation Methodologies

Competitor Usability Testing

Competitor usability testing is observing our target market using a competitor’s products or services to gain insights on the mindset of the user, common issues, and potential improvements in our own product.
Blog Posts & Articles
Validation
Innovation Methodologies

Concierge Test

Concierge is a technique to test the solution of a customer problem by manually performing tasks as a service.
Blog Posts & Articles
Validation
Innovation Methodologies

Demo Pitch

A demo pitch is when we present or pitch our solution using some kind of product or service demonstration in the hope of convincing a potential customer to buy.
Blog Posts & Articles
Validation
Innovation Methodologies

Dogfooding

Dogfooding is simply using the product as if we were the customer and experiencing it firsthand.
Blog Posts & Articles
Validation
Innovation Methodologies

Scorecards

A scorecard is an evaluative decision-making or prioritizing shortcut, often done in a spreadsheet.
Blog Posts & Articles
Validation
Innovation Methodologies

Dashboards

A dashboard is a simple view of product metrics that displays information about the general health and viability of the product.
Blog Posts & Articles
Validation
Innovation Methodologies

Index of Methods

Blog Posts & Articles
Validation
Innovation Methodologies

Net Promoter Score Survey

The Net Promoter Score (NPS) identifies customer loyalty to the brand or product.
Blog Posts & Articles
Validation
Innovation Methodologies

Paper Prototyping

Quickly test a solution idea by simulating user-product interactions using paper drawings and cutouts.
Blog Posts & Articles
Validation
Innovation Methodologies

Wizard of Oz

The Wizard of Oz (WOZ) test involves one or more individuals simulating fully functioning product features without the use of technology.
Blog Posts & Articles
Validation
Innovation Methodologies

Data Mining

Data mining uses statistics from large amounts of data to learn about target markets and customer behaviors.
Blog Posts & Articles
Validation
Innovation Methodologies

Experience Sampling

Get information about a participant's daily behaviors, thoughts, and feelings in real-time, or as close to it as possible.
Blog Posts & Articles
Validation
Innovation Methodologies

Contextual Inquiry

Combination of semi-structured interviews and observations done in the actual location where the problem occurs or the solution will be used.
Blog Posts & Articles
Validation
Innovation Methodologies

Value Proposition Test - Fake Door

People are bad at predicting what they want, but they are great at reacting to new offers. Building just the entry to an offer makes it possible to predict future reactions based on actual behavior.
Blog Posts & Articles
Validation
Innovation Methodologies

Value Proposition Test - Event

We can use an event to smoke test a problem, and to some extent, content that is attractive for our target audience.
Blog Posts & Articles
Validation
Innovation Methodologies

Value Proposition Test - Flyer

Flyers serve as placeholders for conversations that need to happen.
Blog Posts & Articles
Validation
Innovation Methodologies

Value Proposition Test - Pocket

This smoke test applies to physical goods or products with hardware components — at minimum a mobile phone.
Blog Posts & Articles
Validation
Innovation Methodologies

Value Proposition Test - High Bar

A high bar test helps to gauge the customer’s willingness to pay without using any form of monetary payment.
Blog Posts & Articles
Validation
Innovation Methodologies

Value Proposition Test - Landing Page

A landing page used as a minimum viable product evaluates whether a particular type of customer wants a proposed product based on actual behavior (as opposed to a declared preference).
Blog Posts & Articles
Validation
Innovation Methodologies

Value Proposition Test - Pre-Sales

Pre-sales involves accepting payment in exchange for a promise to deliver the value proposition at a later date.
Blog Posts & Articles
Validation
Innovation Methodologies

Value Proposition Test - Sales Pitch

Conducting a sales pitch as a smoke test is when an entrepreneur pitches a value proposition to a potential user or customer for the purpose of gathering feedback on a problem.
Blog Posts & Articles
Validation
Innovation Methodologies

Picnic in the Graveyard

This technique involves exploring "near miss" failures similar to our product idea to generate ideas about what to test and how to build a business model that could work in a second attempt.
Blog Posts & Articles
Validation
Innovation Methodologies

Solution Interview

The solution interview builds on our research around the problem and proposes a solution to see how potential customers react.
Blog Posts & Articles
Validation
Innovation Methodologies

Index of Questions

Blog Posts & Articles
Validation
Innovation Methodologies

Product/Market Fit Survey

The Product/Market Fit Survey (PMF) specifically asks the customer, “How would you feel if you could no longer use this product?” with a multiple choice response.
Blog Posts & Articles
Validation
Innovation Methodologies

Usability Testing

Usability testing involves observing people trying to complete a series of tasks while using an interactive product.
Blog Posts & Articles
Validation
Innovation Methodologies

A/B Testing

A/B testing (also known as split testing and bucket testing) is a randomized method of comparing two versions of an element (A and B) against each other to determine which one performs better using a metric to define success.
Blog Posts & Articles
Validation
Innovation Methodologies

Off-Brand Testing

Off-brand testing is a technique that helps established companies run experiments while minimizing risk to their brand equity.
Blog Posts & Articles
Validation
Innovation Methodologies