Pay no attention to the man behind the curtain- Wizard of Oz Prototype Example

Wow I’m tired.

Tired and energized.

In the past few weeks we’ve had the typical ups and downs of every startup. We’ve let in a number of alpha users, got another hundred signed up thanks to a very minor TechCrunch mention, and realized our Minimum Viable Product wasn’t minimum enough.

We started building our site with a basic premise that entrepreneurs were willing to post their pitches to attract co-founders and investment.

We confirmed this hypothesis through numerous customer interviews and surveys.

We iterated on nothing more than Photoshop mockups.

We test drove our site with a limited number of alpha users and despite the clunky interface we were pleased by the results. We pushed people through a rather long survey and got an 82.9% completion rate, far higher than we would’ve thought.

So we knew that people were willing to fill out some forms.

So you can imagine our surprise when our first chunk of 50 alpha users decided not to post any business ideas or pitches. Oops.

Behavior Matters

We now realize that social proof and user experience is going to be more and more critical for us.

Without a few hundred people posting ideas and pitches, new people to the site won’t be bothered to post their own. A classic chicken and egg problem which makes our alpha testing more difficult.

Also a classic confirmation of the behaviorist maxim:

I don't care what you say, I care what you do. Click To Tweet

wizard of oz prototype example

So aside from pondering whether or not our minds are epiphenomenal to our behavior (yes, I’m a philosophy geek), where does that leave us?

Without getting a basic sense of how many people are willing to put in data, we can’t adequately test our other assumptions regarding interaction with the website.

So we need to get more basic.

Pay No Attention

pay-no-attention-to-the-man-behind-the-curtainAardvark allegedly ran for 9 months in Wizard of Oz prototype example mode. Meaning that there were a number of employees sitting around looking up answers to questions manually and punching in the answers. A number of other services seem to do this as well using Mechancial Turk or a small army of outsourcers.

The point of doing things that way is to see if someone is willing to use or pay for your service before you waste a lot of time building something obscenely complicated. We thought we were doing that by testing our assumption with customer interviews, mockups, and alpha testing.

We could have done less. Click To Tweet

So we’ve decided to try and simplify even more and focus on the most basic type of co-founder matching. We’ve got a few hundred entrepreneurs. We’re going to match them up one by one.

It may take a while.

We’re doing this to test out a basic assumption that we skipped before. We always asked if entrepreneurs were willing to post their ideas. What we forgot to ask is, “Is anyone willing to read them?”

So far I’ve got a 90% “Yes”. Still, “fool me once…”

We’ll test behavior instead of survey responses by sending out pitches via email, directly to people who’ve expressed an interest. We’ll be measuring how many people open the mails and how many people actually click through and read the full story. We hope we’ll find the unsubscribe rate is as low as our monthly mailer (<1%).

I’ll settle for <10%. 🙂

Got a pitch? Want in on the action? Go to our home page and signup now!

[UPDATE: The startupSQUARE is closed. Read why here.]

Make better product and business decisions with actionable data


Gain confidence that you're running the right kind of tests in a five-week series of live sessions and online exercises with our Running Better Experiments program. Refine your experiment process to reduce bias, uncover actionable results, and define clear next steps.
Reserve a seat