The long market research survey (anything that takes over 20 minutes to finish) is a dead man walking. Just like a colleague you all know is going to be fired, long surveys are headed towards the insight scrap heap.
Most customers today are not prepared to spend over 20 minutes filling in a survey, and studies have shown that the people that answer long surveys are significantly different from the typical customer. They are heavy users of the internet, and are more likely to buy and browse online. They don’t represent your customers or the population.
Surveys have also become less suitable to the tasks of understanding customers as markets have become more complex. Today’s business landscape involves more brands, more product lines and more channels to reach customers. When companies use surveys to understand their complicated market, the surveys get longer and more tedious. Sending a survey about 20 competitors, as a typical company needs to do to understand its market, will only frustrate your customers.
Most importantly, long surveys are not suitable for mobile devices. About 30 percent of all online surveys are now completed from a mobile device, and this is expected to increase to over 50 percent within two years.
If you are still using long market research surveys, you are wasting money, collecting inaccurate information, annoying customers and missing out on the opportunity to gather impactful business insight.
The following points highlight some of the evidence that long surveys are past their sell-by date.
- Declining popularity. The most recent ESOMAR Global Market Research report (2014) showed a decline in the use of surveys, as a proportion of all money spent, and this is predicted to fall further.
- Declining response rates. The gold standard for surveys is the Pew Research Center and they report that response rates have fallen from 36 percent in 1996 to nine percent in 2012. They are expected to keep falling.
- Satisficing is when a respondent takes the quickest route through a survey to gain prizes and other incentives, for example by claiming to buy fewer products than they really buy, and by straightlining their way through studies. Longer surveys have been shown to be more likely to suffer from satisficing. For instance, a 2011 study of 100,000 surveys showed that as surveys get longer, people spend less time per question.
- People aren’t grids! Over the past ten to 15 years, there has been more research that confirms that people are essentially emotional and that they employ time-saving patterns to their decision making. For example, Nobel Prize winner Daniel Kahneman, author of the best seller Thinking, Fast and Slow, has shown that about 95 percent of decisions are made by an automatic process, based on habit, emotions and patterns—a process he refers to as System 1. He shows that System 2, what we would consider to be rational thinking, is only used for a relatively small number of decisions. However, a long survey is a System 2 sort of exercise because it asks research participant to reveal the multiple factors that contributed to their decision (for example, by completing a grid of questions about brands and attributes).
Ten years ago, there were two main options for finding out what customers wanted: the survey and the focus group. Now there is a long list of alternatives, and the list is growing ever longer. Here are three themes that underpin the replacements for long surveys (and increasingly the alternatives for focus groups):
- Ongoing discussions. Rather than trying to ask people every question in one go, research is becoming more like real life. I ask a question, you say something, and I listen. Then maybe I ask another question, based on what you have said, or perhaps you ask a question. This theme underpins approaches such as insight communities and customer engagement. The learning is continuous, respectful and focused on what can be achieved. This process tends to be a mixture of discussions and forums.
- In the moment. In the moment is about asking people questions at the time that something is happening, for example when they visit a store or when they use a product, while the experience is still fresh. I see this theme manifest itself in many mobile research approaches, especially those where participants gather ethnographic information in the form of photos, diary entries, and video.
- Observational data. Observation is used at large and small scale. At large scale we often refer to it as big data (for example looking at media consumption, transactions, or social media). At the smaller scale it is often related to creating experiments, to measure what happens when A is changed for B.
The Big Picture
Many people still use long market research surveys. However, these projects continue only because companies have done them that way for a long time. Many of the people supplying long surveys are on their way out.
Long market research surveys do not work well, are not delivering value, are not delivering insight, and are certainly not delivering impact.
Where should you start? If you have any surveys over 20 minutes, they need to go. Don’t just try to trim them. Consider other alternatives and look to provide better experiences for your customers. Think mobile, think engagement, and think about developing long-term relationships with customers.