Wizard of Oz
Wizard of Oz
In Brief
The Wizard of Oz (WOZ) test involves one or more individuals simulating fully functioning product features without the use of technology. All inputs, outputs, and algorithms are performed manually by humans, but without the knowledge of the customer. This allows entrepreneurs to test a complete solution and gain accurate customer feedback without the cost of building a complex, fully automated solution. It is often confused with Concierge Testing.
Helps Answer
- Which features will provide the best solution?
Tags
- B2C
- B2B
- Quantitative
- Value proposition
Description
The WOZ test is a well-known experiment from the 1970s. It involves one or more humans, so-called wizards, who manually simulate the functionality of a product. However, the wizards are hidden so that the user does not know that someone is manually doing the work.
Time Commitment and Resources
WOZ is not a "quick and dirty" prototyping method, as it requires a high commitment of resources. Depending on the test, there can be more than one wizard interacting "behind the curtains" with the customers. The time commitment also heavily depends on the task domain and the number of users exposed to the product. For instance, categorizing images by a wizard is easier and faster than transcribing voice for speech recognition. Hence the time commitment can vary from days to several weeks.
How to
- Build a prototype of your product without the fancy algorithms/technology.
- Allocate at least one person (the “wizard” behind the curtain) who simulates the interactive behavior of the product.
- Show your product to your users and let them use it.
- Collect data to see if your product delivers value to your users.
Interpreting Results
The WOZ test can provide valuable information, especially quantitative data. You can also gather information about the nature of interaction with your product. Summarize all observations and use the insights to adjust your solution accordingly.
Potential Biases
- Sampling bias: See Concierge Test
- Confirmation bias: See Concierge Test
Field Tips
- Be lazy. If you can hire five interns to fake a feature, don’t bother coding it. - @TriKro
- The Wizard of Oz method is for evaluating a solution hypothesis. - @TriKro
- Pareto Principle + Wizard of Oz Experiment is all you need to invest your time in a successful business - @andreftavares
- Got a tip? Add a tweetable quote by emailing us: [email protected]
Case Studies
- Fast Company: Cardmunch
- Groupon: 10 Massively Successful Minimum Viable Products
- Harvard Business Review: Aardvark
- Zappos: Wizard of Oz test
- Got a case study? Add a link by emailing us: [email protected]
Tools
- Got a tool to recommend? Add a link by emailing us: [email protected]
References
- GrasshopperHerder: What Type of Lean Startup Experiment Should I Run?
- Roskilde University—Centre for Cognitive Science: Wizard of Oz Prototyping: When and How?
- Y Combinator: The Wizard of Oz Approach
- Quora: What Are the Pros and Cons of "Wizard of Oz Testing"?
- Usability Net: Wizard of Oz
- Got a reference? Add a link by emailing us: [email protected]