5.5 Concierge Test
At a Glance
Other names Concierge · Concierge MVP
In Brief
In a concierge test you deliver your product’s value by hand, as a personal service, to a small number of real customers. The defining move is that a person — usually you — performs every step end-to-end, solving the problem directly, and gathers feedback on what works, what does not, and which features are essential. The output is qualitative insight into the minimum feature set you need before you invest in building an automated product.
Common Use Case
You believe your product idea will save customers time or effort, but you haven’t tested the actual workflow yet. Instead of building anything, you personally deliver the service by hand for a handful of customers, learning exactly which steps matter, what they actually need, and whether they would pay — all before you write any code.
Helps Answer
- Does this solution actually solve a real customer problem?
- How can the problem be solved in practice?
- What is the minimum feature set needed to deliver the solution?
- What are the biggest obstacles to delivering this solution?
Description
In a concierge test you deliver the value proposition as a service. Like a hotel concierge, the work is highly customized and customer-facing: you perform the tasks by hand, usually for only a few customers, because it is not cost-efficient to scale. The close contact gives you detailed feedback from the segment you care about, and because nothing is built, you can adjust the service from one customer to the next at very low cost.
When a physical or analog artifact stands in for the software instead of a person performing the service, that is an Analog/Digital test.
Start from a clear value proposition. Where a Solution Interview asks customers whether a proposed solution matches their needs, a concierge test puts the solution in their hands and watches what happens. Design the service around the steps a customer would take with your eventual product, and build it around the leap-of-faith assumptions you most need to test.
Deliver the service manually and personally, starting with a small batch so you are not overwhelmed. You do not need any code or automation at this stage. While delivering, keep collecting feedback and adjust the service as you go. Over time you learn what customers expect and what they actually value, and you can gradually automate the parts that work. Stop expanding the manual service once it stops producing new major insights — a concierge test is a phase, not a permanent way to operate.
A concierge test is distinct from a Wizard of Oz test: in a concierge test the customer knows the service is delivered by a person, whereas in Wizard of Oz the manual work is hidden behind an interface that appears automated.
How to
Prep
- Write down the value proposition you are testing. Be specific: “We help [customer] do [task] so they get [outcome].” This is the claim your concierge test will validate or invalidate. If you cannot finish that sentence, you are not ready to run a concierge test — you are still in problem discovery, and a Customer Discovery Interview is the better next step.
- Pick the segment. Concierge tests do not scale, so the segment you pick determines what you can learn. Choose a tightly-scoped early-adopter group: people who already feel the pain acutely, can describe their current workaround, and are reachable without a marketing budget. The wrong segment produces friendly-sounding feedback that does not generalize.
- Map the manual flow. List every step the customer will go through — the same steps they would go through with your eventual product. For each step, decide how you will deliver it manually: email, spreadsheet, phone call, in-person meeting, or a shared document. The flow you draft here is the spec for what gets automated later, so write it as if a junior teammate would have to follow it.
- Identify which steps to automate first. Tag each step on the manual flow as must stay human (the part where the value proposition lives), automate first (high-volume, low-judgment, easily templated), or automate later (judgment-heavy but eventually scriptable). The automate-first list is what you watch the test against.
- Recruit 3–5 early adopters. Find people who already have the problem and are willing to try your service. Ask for payment (even a small amount) — paying customers give more honest feedback than non-paying ones.
- Set expectations. Tell customers this is an early, hands-on version of the service. You will be personally involved, things may be rough, and you want their honest feedback.
Execution
- Deliver the service manually. Perform every task yourself. Track how long each step takes, where customers get confused, what they ask for that you didn’t anticipate, and what they don’t use.
- Collect feedback after each interaction. Ask: “What was most valuable? What was frustrating? Would you pay for this again?” Record responses immediately.
- Adjust and repeat. After each batch of interactions, update your service based on what you learned. Change pricing, add steps, remove steps. The concierge test is iterative — you are designing the product through direct delivery.
- Know when to stop. When you stop hearing new feedback and can predict what customers will say, you’ve learned what you can from manual delivery. Begin automating the parts that work.
Analysis
- Aggregate across customers. The data is mostly qualitative. Pull together notes from every customer, grouped by each part of the service — time per step, where they got confused, what they asked for, what they ignored — so you can see patterns rather than one-off reactions.
- Identify the high-value and the automatable steps. Separate the steps where the value proposition lives (the part customers are paying for) from the high-volume, repeatable steps that are candidates to automate first. Look for steps that take the most time and recur most consistently across customers.
- Surface obstacles. Note where the manual delivery broke down or surprised you. These are the implementation problems your eventual product will have to solve.
- Feed the insights back into the service. Adjust pricing, steps, and scope based on what you learned, then keep delivering until the feedback stops producing new major insights.
- Concierge halo (false-positive bias) The manual delivery adds value of its own: trust, responsiveness, and a personal relationship. Founders can mistake positive feedback on the hand-delivered service for validation of the product concept. When you automate, that human layer disappears and customers may reject the result. Separate feedback about the outcome from feedback about the personal touch, and treat enthusiasm that depends on you being in the loop as unproven until automation is tested.
- Sampling bias (unrepresentative batch) A small or homogeneous batch produces feedback that feels definitive but does not generalize to the broader target segment. If the first customers you can recruit share an unusual trait — same company, same network, same tolerance for rough edges — their reactions may not predict how the wider segment responds. Actively recruit from different sub-groups within your target segment to stress-test what you are learning.
Learn more
Case Studies
Airbnb: Air mattresses in a San Francisco loft
Brian Chesky and Joe Gebbia photographed their San Francisco apartment, posted it for design-conference attendees, and hosted three paying guests to test whether people would pay to stay in a stranger’s home.
Amazing Airfare: $8 manual deal alerts
Zachary Cohn validated his flight-deal alerts concept by collecting $8 each from five subscribers via PayPal and emailing them deals from a spreadsheet before building any software.
GE Healthcare: Concierge-style infant monitoring trial
GE Healthcare tested a new infant monitoring concept in emerging markets by deploying trained nurses with portable tools to deliver the experience by hand, validating workflow fit and perceived value before investing in complex diagnostic equipment.
Food on the Table: Concierge meal-planning
Manuel Rosso recruited Food on the Table’s first customers by accompanying Austin grocery shoppers on their trips and emailing personalized meal plans, learning the workflow by hand before any software was written.
DoorDash: PaloAltoDelivery.com
The founders launched a bare-bones site listing local restaurant menus and personally took the orders and delivered the food themselves, learning the logistics by hand before building any dispatch software or incorporating as DoorDash.
Got something to add? Share with the community.
