4.4.2 Value Proposition Test - Broken Promise
At a Glance
Other names Broken Promise Test · Virality Test
In Brief
The broken promise smoke test measures how much natural word-of-mouth a product idea has before anything is built. You share a landing page with a small group of 10-15 people and explicitly ask them not to tell anyone, then track whether new sign-ups appear from people outside that group. The signal is a referral rate: if sign-ups exceed the number of people you contacted, the idea was compelling enough that people broke a social promise to share it.
Common Use Case
You have three different product ideas and limited time to build. You create a simple landing page for each, share it with a small group of contacts, and ask them not to tell anyone. A week later, one page has sign-ups from people you never contacted. That idea has natural word-of-mouth potential worth pursuing.
Helps Answer
- Does this idea address a need people feel strongly enough to share?
- Are people excited enough to tell their friends about it?
- Which of several ideas generates the most organic sharing?
- What kind of person gets most excited about this concept?
- Is there natural word-of-mouth potential before we build anything?
Description
A broken promise smoke test shares a real-looking product idea with a small, hand-picked group, asks them explicitly not to tell anyone, and then watches for sign-ups from people outside that group. Each outside sign-up is a broken promise: someone found the idea compelling enough to pass it along despite being asked to keep it quiet. The commitment it tests is social rather than monetary — that places it among the lower-commitment rungs of the Value Proposition Test family — and the signal it produces is a referral rate measuring organic word-of-mouth before anything is built.
The mechanics work like a Landing Page Test: a dedicated web page with a sign-up button. The difference is the explicit request not to share. On the thank-you page or in follow-up communication, the copy asks the person who signed up NOT to tell others about the idea. After a set period, you compare the original list of people you contacted against the full list of sign-ups. Anyone who was not on your original list is counted as a referral — they heard about it from someone you seeded.
The measure is a referral rate, not a conversion rate. (A conversion rate is the share of people you reached who took the action; a referral rate counts sign-ups against the people you seeded, so it can exceed 100%.)
Referral rate = total sign-ups in the period ÷ number of people originally contacted
A rate above 100% means more people signed up than you reached directly, so the idea spread on its own.
This test suits early-stage founders comparing several ideas at once. It helps identify which ideas have enough pull that people pass them along, so you can focus on the one most worth building.
Note who is being referred where you can. A referral chain that reaches the right kind of prospect tells you more about the target profile than the rate alone, and those referred prospects are worth interviewing to learn what drew them in.
Some founders use AI-simulated personas to pre-test messaging before running the real experiment — a useful sanity check for obvious weaknesses, but not a substitute. The point of this smoke test is to measure real human behavior: whether real people break a real social promise because they find the idea too compelling to keep secret. AI cannot simulate genuine social pressure or authentic word-of-mouth enthusiasm.
How to
Prep
1. Define the “promise” and the ask not to share.
Decide what exclusive access you’re offering (early invite, beta seat, founding-member status, a private waitlist) and write the explicit ask not to share. Both halves matter: the offer creates the social capital, and the explicit “please keep this to yourself” is what makes a downstream signup count as a broken promise rather than just a forwarded link. Draft the language you’ll use on the thank-you page or follow-up email so the request is unambiguous.
2. Identify and recruit a representative seed group.
Pick 10-15 people who actually look like your target prospect, not friends and family who already love everything you do. Mix in people you know less well — colleagues of colleagues, members of relevant communities — so the network downstream of each seed contains real prospects, not just your existing fan club. Keep an exact list of who you contacted; you will need it to count referrals later.
3. Pre-commit to a referral-rate threshold and pick a traffic source.
Decide before launch what referral rate constitutes a go signal (e.g., signups exceeding 100% of seeds, or a specific count like 30% of seeds producing referrals). Picking the cutoff after you see results is how confirmation bias creeps in. Also decide how you’ll attribute traffic — a unique share link per seed (a tracking tag in the URL that records who the visitor came from), a separate landing page per seed, or an explicit “how did you hear about us?” field — so you can tell referrals apart from the original group.
4. Stand up the landing page and tracking before you send anything.
Have the page, signup form, attribution mechanism, and a way to compare “list of seeds” vs. “list of signups” all wired up in advance. Running the test for a week and discovering you can’t tell who came from where is the most common way this experiment fails.
Execution
-
Send the seeded outreach once per seed. Reach out to your 10-15 seeds individually, each with their own attribution tag or landing-page link wired in. Use the promise copy you wrote in Prep, including the explicit request not to share. Send once — do not follow up with non-responders, since a non-response is part of the signal.
-
Watch the sign-ups, not the seeds. Track sign-ups as they arrive and record which attribution tag each one carries. Sign-ups with no seed tag are the ones to watch — they came from outside the group you contacted.
-
Stay hands-off for the full window. Do not nudge seeds, send reminders, or coach anyone on whether they may share. Any prompting contaminates the result; silence is data. If a seed asks whether they can share or posts the page publicly, note it but do not advise them.
-
Log the spread. At the end of the window, you want a clean count of who came from each seed and how many arrived with no seed tag. Those untagged sign-ups are the broken promises — people who shared an idea they were asked to keep quiet.
Analysis
-
Compare against your pre-committed cutoff. Read the result against the referral-rate threshold you set in Prep, not one you pick now that you can see the numbers. The cutoff is what turns the rate into a go or no-go decision.
-
Decide exactly who counts. Be clear on who belongs in the original group and who counts as a referral. Immediate family — your mom, and anyone she refers — does not count unless she genuinely represents your target market. Excluding non-representative contacts on both sides keeps the rate honest.
-
Calculate the referral rate. Divide total sign-ups in the window by the number of people you originally contacted, separating genuine referrals from the seeds. Flag any sign-ups whose source you cannot attribute cleanly rather than guessing which side they fall on.
-
Read the result against your sample size. With only 10-15 seeds, a few extra sign-ups swing the rate a lot. Treat a strong result as directional evidence to act on, not a precise measurement, and let the size of the gap above your cutoff set your confidence.
- Confirmation bias Founders tend to seed people who already love the idea, which inflates the apparent referral rate. Recruit a seed group that looks like your target prospect, not just enthusiasts who will share anything you make.
- Moving-the-goalpost bias Setting or adjusting the referral-rate cutoff after you see results is how a weak signal gets reframed as a green light. Fix the cutoff in Prep, before you run the test, and read the result against it unchanged.
- Survivorship bias on referrers A handful of enthusiastic sharers can mask broad indifference. Two loud advocates are not the same as a widely loved idea, so judge by the overall referral rate rather than by the most active sharer.
- Attribution error When you cannot tell a referred sign-up from one you reached directly, it is easy to credit the idea for traffic it did not earn. Wire up per-seed share links or a “how did you hear about us?” field in Prep, and exclude any sign-up you cannot attribute rather than assuming it was a referral.
- Gaming and self-referral A motivated founder can pad the count by signing up through alternate accounts or asking friends to. Decide up front which contacts are excluded from both the seed and referred counts, and treat any traffic spike that does not resolve to a real source as suspect, not as success.
Learn more
Case Studies
Further reading
- The Art of Cleverness — Mike Chen, “Use the Broken-Promise Strategy to validate your startup”
- Andrew Chen, The Cold Start Problem, HarperBusiness, 2021 — scarcity, waitlist, and invite-only mechanics in early viral growth (Clubhouse and other case studies).
- Steve Blank — Customer Development Manifesto
- Eric Ries (2011) — The Lean Startup, Chapter 6 “Test”
- Tim Ferriss blog — Jeff Raider on Harry’s pre-launch campaign
Got something to add? Share with the community.
