6.5 Usability Testing

In Brief

Description

Usability testing is a structured observation method where you put a representative user in front of your product, give them a realistic goal to accomplish, and watch where their behavior diverges from what the design intended. You record whether they finish the task, where they hesitate, what they misread, and how they feel along the way. The aim is to find the points where the design confuses real people before you ship it to all of them.

The method works at any fidelity. A paper sketch tested with three people can expose a confusing label as effectively as a fully built product tested with twenty. Small, frequent tests run by the team that built the product close the loop faster than commissioning occasional large studies, and you do not need a big sample: five carefully chosen testers, run in iterative rounds, will surface most of the dominant problems in a flow. Usability testing tells you what breaks and why; pair it with methods that tell you how often, such as analytics or A/B Testing.

Where remote platforms are available, AI is an optional accelerant for the slow parts: it can generate session transcripts, flag moments of hesitation or frustration from screen and audio signals, cluster patterns across sessions, and draft summaries, keeping analysis to minutes and making it practical to run more tests with more users. It accelerates synthesis but does not replace observation — a flagged frustration moment still needs a human to decide whether the design is wrong or the task itself was unfair, and watching a user struggle live builds an understanding that a transcript summary cannot reproduce.

How to

Prep AI Prompt

1. Define the task scenarios. Write 3–5 realistic tasks that cover the critical paths in your product. Each task is a goal, not a click-by-click instruction. Example: “You just signed up. Find the feature that lets you invite a teammate.” Avoid leading the user toward the answer. Set a reasonable time limit per task (2–5 minutes).
2. Recruit 5 testers who match your target user. Five is the rule of thumb — past five, you mostly re-confirm problems you have already found. Recruit from existing users, social channels, or online research panels. For B2B, ask users directly; most will say yes if you make it short.
3. Decide moderated vs unmoderated, then pick the tool. Moderated sessions (video call or in-person) let you ask “what made you click that?” in real time but cost more researcher hours. Unmoderated sessions on remote testing platforms scale cheaply, and AI-assisted platforms can analyze think-aloud audio and behavioral signals across many sessions without a researcher watching each one in real time. Mix both when budget allows: a few moderated sessions to build empathy, a larger unmoderated batch to confirm patterns.
4. Write a short intro script. Two or three sentences that frame the session. The standard reassurance: “We’re testing the product, not you. If you get stuck, that’s exactly the feedback we need.” Pre-writing the script keeps facilitators consistent across sessions.
5. Pilot once internally before the first real session. Run the full protocol with a teammate. You’re checking whether tasks are intelligible, whether the recording setup works, and whether the timing fits. Half the issues you find in piloting would have wasted a real participant slot.
6. Decide who moderates and who takes notes. One person facilitates and asks the questions; a second person observes and captures notes. Trying to do both at once degrades both. For unmoderated tests, the “observer” role becomes whoever reviews the AI-generated session summaries and watches the flagged moments.

Analysis AI Prompt

1. Synthesize observation notes across sessions. The facilitator and any observers compare notes, focusing on moments where users hesitated, backtracked, or expressed frustration. Even usability experts sometimes disagree on interpretation, so multiple observers reduce the chance that a single experimenter’s bias filters the findings. Identify the problems that affected most or all participants — those are the ones worth fixing first.
2. Prioritize by frequency and severity. Given the small sample size, treat consistent problems (3+ of 5 users hit the same wall) as confirmed defects. One-off issues might be real or might be participant-specific — note them but do not over-weight them. Severity is independent of frequency: a problem that happens once but blocks the user from completing the task scores higher than a cosmetic issue that everyone hit.
3. Use AI tools to triage what to watch closely. AI session-analysis tools auto-flag moments of confusion, generate heatmaps (a color overlay showing where users clicked, tapped, or looked most), report how many participants finished each task and how long it took them, and produce session summaries. Treat these as a way to triage which recordings to watch closely, not as a substitute for watching sessions yourself. Automated checks for contrast, touch-target size, and navigation depth can also catch surface-level issues before you spend a real participant slot on them.
4. Distinguish usable from desirable. If all users complete the tasks, that tells you the product is usable. It does not tell you whether the value proposition lands or whether anyone would actually pay for it. Usability testing is a necessary but insufficient validation step — pair it with desirability and value-proposition tests before you assume you are clear to ship.