Experience Sampling
In Brief
Experience sampling (also called ESM or daily diary method) is a research technique that prompts participants to record their behaviors, thoughts, and feelings at set intervals throughout their day while they are in their natural environment. You send brief prompts via text, app notification, or email, and participants respond in the moment. The output is real-time qualitative data on how often a problem occurs, what triggers it, and how people actually feel when it happens — free from the distortions of retrospective recall.
Common Use Case
You believe a problem exists for your target customers, but you don’t know how often it actually happens or how they feel about it in the moment. You send participants brief prompts throughout their day so they report what is happening in real time. The responses reveal frequency, intensity, and emotional context that retrospective interviews would miss.
Helps Answer
- Who is experiencing this problem regularly?
- What frustrations or positive moments come up during the day?
- How often does this problem actually occur?
- Where and when do potential customers encounter this issue?
Description
The key to experience sampling is asking the right questions. Be especially careful with phrasing, since you will be asking the question over and over again. This method makes most sense when you want to solve a frequently reoccurring problem. You get the most useful and viable input when asking about repeated behavior, and more specifically, the last time it occurred.
Reed Larson and Mihaly Csikszentmihalyi introduced the Experience Sampling Method in 1983 to capture moment-to-moment psychological states without the distortions of retrospective recall, and Hektner, Schmidt, and Csikszentmihalyi later expanded it into a practitioner handbook (2007). The method is framework-neutral — researchers in HCI, mobile health, and consumer behavior all use it — but the moment-of-experience sampling logic is the constant.
How to
Prep
Preparation:
- Carefully phrase the question.
- Make sure the answering process takes no more than a minute.
- Plan how often you want to send alerts, both how many times a day and distribution over the days. Be careful that the frequency doesn’t lead to the perception of being nagged. If the user hasn’t completed the behavior, another alert may create an undesirable effect.
- Choose your medium of contact — SMS, phone, email, app, etc.
- Plan how to collect the data; a spreadsheet is common.
- Decide how many participants you want and start recruiting as soon as possible.
- Plan the analysis according to the expected amount of data, team size, process, etc.
Finding Participants:
- Use a screener to select relevant participants.
- Identify participant criteria and formulate questions accordingly. If possible, use quantifying questions (e.g., how often the participant does something).
- Consider non-criteria that your questions might not cover yet.
- Check willingness to participate by collecting contact information.
- Check availability.
- Select your participants.
- Set their expectations according to how often they will be asked to give answers.
AI-assisted prompting: Conversational AI chatbots (via SMS, WhatsApp, or in-app messaging) can replace rigid alert-and-form workflows with natural-language prompts that feel less intrusive and yield richer responses. Consider this approach when participants are likely to disengage from traditional form-based pings.
Execution
Once your protocol is set, the in-field phase is mostly about not getting in your participants’ way:
- Send your first wave of prompts on schedule and confirm the medium is working (delivery receipts, app notification permissions, completed first responses).
- Watch the early returns. If responses are sparse or uniformly thin, intervene now — adjust the prompt wording, the time-of-day distribution, or the question count before the rest of the data lands.
- Remember to thank participants after each response. A short acknowledgement preserves engagement across the rest of the sampling window.
- Resist the urge to coach. If a participant asks what a question means, note the confusion as a wording issue rather than rephrasing it for them — the rephrased version is no longer comparable to the rest.
- Collect the data into your analysis surface (spreadsheet, ESM platform export, or research repository) as it arrives. Letting it pile up until the end forces you to do triage and analysis simultaneously.
Analysis
- Check the first set of answers to see if they are sufficient for your research. If necessary, expand your questions or explain to participants the level of detail you need.
- Check if the questions are correctly understood. If necessary, adjust your questions or correct individual participants.
- Minimum viable study: 10 participants × 10 prompts each gives you 100 data points — enough to spot themes. With fewer than 50 total data points, treat findings as hypotheses for follow-up interviews, not conclusions.
- Begin the analysis as soon as possible; do not wait until you have collected all the data.
- Eyeball the data to get a general impression.
- Decide on categories to help you organize the data.
- Adjust categories during the process if necessary — split if too big, combine if too small.
- Clean the data of answers that are not useful as you run across them.
- If you analyze in a team, work on the first 50–100 data points together, deciding on categories and classifying the answers.
- Distribute the remaining data among the team for classification; answers may match multiple categories.
- Switch the data within the team for a second blind classification and discuss possible discrepancies.
- Create frequency charts.
First, look at the frequency distribution and identify common themes to gain insight into participants’ pain points and delights. Then pinpoint what you have and have not been doing well in solving your target group’s problems, as well as opportunities for improvement. You may find that the problem is slightly different than expected, or what you thought was a problem is not one at all. You may get ideas for additional product features. In any case, you end up with data on different experience categories and therefore many opportunities.
Tools like Dovetail and Marvin can automatically extract themes, detect sentiment shifts, and flag outlier responses that deserve deeper investigation. What previously required a team spending hours on affinity mapping can now be drafted by an LLM in minutes, letting researchers spend their time validating AI-generated themes rather than building them from scratch.
-
Prediction bias Do not ask about people’s opinions on potential products, situations, or what they think they need. People are bad at predicting the future. Ask about recent behavior and problems.
-
Confirmation bias Be careful not to use leading questions or give examples of what kind of answers you expect.
-
Retrospective recall bias The whole point of ESM is the in-the-moment response. If a participant batches their answers at the end of the day, that entry has the same recall distortion as a traditional interview. Flag late responses in your data and consider weighting them differently.
-
Survey fatigue Compliance drops as the sampling window goes on. Watch for thinning responses and consider shortening the question list or shortening the window rather than pushing through.
-
Scale bias AI makes it tempting to scale ESM studies to thousands of participants, but larger sample sizes amplify the risk of shallow or low-quality responses. Prioritize well-screened participants and carefully phrased questions over sheer volume.
-
Run a comprehension test before a landing page test or you won’t understand why it doesn’t work. - @TriKro
-
Don’t ask for opinions, observe behavior. - @tsharon
-
Often, what customers say they want and what they actually need differ significantly. - @macadamianlabs
-
Trying to understand users without actually observing them is the same as learning to ride a bike by reading about it. - @MarkusWeber
Learn more
Case Studies
Using experience sampling methodology to understand how educational leadership students solve problems on the fly
Spotify
Spotify combined a 3-week diary study with behavioral log data tracking on the same group of listeners. The discrepancies between self-reported experience and actual behavior proved to be the most valuable insights, directly informing ad product design.
Headspace
In a randomized controlled trial, Zawadzki and colleagues used ecological momentary assessment to study stress reduction among novice meditators using the Headspace app. The protocol ran 4-day measurement bursts of five prompts per day at baseline and follow-up waves at weeks 2, 5, and 8 — revealing that improvements in mindfulness appeared earlier than traditional pre/post survey designs could detect.
Further reading
- Larson, R. & Csikszentmihalyi, M. (1983). “The Experience Sampling Method.” New Directions for Methodology of Social and Behavioral Science, 15, 41–56. (Originating paper.)
- Hektner, J. M., Schmidt, J. A., & Csikszentmihalyi, M. (2007). Experience Sampling Method: Measuring the Quality of Everyday Life. Sage Publications. ISBN 978-1412949231.
- Don’t listen to users, sample their experience — Tomer Sharon (Video)
- Experience Sampling Method — Wikipedia
Got something to add? Share with the community.