Fixed set of questions, usually via a mass-communication channel such as an email or website pop-up.
An open-ended survey asks a fixed set of questions, usually via a mass-communication channel such as an email or website pop-up. Answers are not constrained as in the case of multiple choice or check boxes, but are free-text responses in which the customer can choose the length and detail of their answer.
Surveys can be quick to write and execute, often taking only 1-2 hours to set up. However, designing effective questions that don’t introduce biases into customers' answers can require a high degree of skill, multiple revisions, and even comprehension tests run on the survey.
Collecting results typically takes more time and depends on the communication channels available to distribute the survey. Response rates can vary from 1 to 20 percent on a survey sent to existing customers, depending on the level of customer engagement, so large numbers of target customers and a good deal of time may be required to collect data.
Analyzing the data can take 4-8 hours, depending on the length of the survey, the number of respondents, and the quality of responses. As answers are free text, a large amount of reading, transcribing, and synthesizing may be required.
An open-ended survey is a generative research technique, and as such, be careful to interpret any input as simply ideas, not as a vote from the customer. The data is qualitative in nature.
Because surveys are flexible, easy to write, and easy to deploy, they are more likely to be misunderstood and misused. Surveys are often a default research method when researchers do not feel they have the time to conduct ethnography or customer interviews. They are often highly favored in a corporate setting because a large number of respondents may be considered statistically significant, even if the survey responses are qualitative.
Open-ended surveys are sometimes combined with closed-ended surveys, making it tempting to spend extended periods of time analyzing the data and looking for correlation that will draw a definitive conclusion to act upon. This tendency to use the data to drive a firm conclusion even when the data is generative in nature is the biggest argument for avoiding surveys at all costs.
A typical debrief method to analyze the generative data is to read each answer and transcribe salient points on post-it notes for a sorting exercise. Patterns can then be more easily identified.
For surveys that specifically solicit suggestions from users, the entire list of suggestions may be added to a repository for later analysis.
In the case of very large data sets, algorithmic tools such as sentiment analysis or word clouds can give additional quantitative insight, but should be used to supplement the qualitative insights, not replace them.