The flow of research

How to address 3 major challenges in qualitative research online

January 31, 2013

The best practices in qualitative research are well established. Faced with a qualitative research challenge, most “qualies” would naturally take the funnel down approach.

Take a hypothetical qualitative research need for testing some new tea flavours. Here’s how a typical funnel down approach would generally shape offline, with in-depth interviews or focus group discussions:

  • 1. Focus on understanding how consumers feel about tea overall.
    2. Explore how consumers feel about different types of teas, fit with occasions, and how it makes them feel.
    3. Finally, show and test the potential concepts.

Throughout this flow, we probe with the ever-pertinent ‘why’ of qualitative research. This approach works well because when analysing the data there is little to no bias introduced during the process. We understand the associations to tea and perceptions on flavours in tea before we have asked about the concepts we want to test.

If we were to do the same qualitative study online with an asynchronous live chat we could pretty much follow the same approach we’d use offline.

But what if we want to explore this through an online forum or discussion board? As soon as we move away from real-time, start-to-finish conversations – or simulations thereof – we face the possibility that bias will be introduced during the process. We also face potential problems like respondent disengagement, or participant drop-out.

To avoid these outcomes, we need to address three key challenges in forum research, each of which presents its own challenges to data quality:

1. Challenge: Tapering participation. Irrespective of how you incentivise, the response rates with discussion forums almost always taper with time. Sometimes it is the topic, or people get busy, or they have said everything they had to say on day one of the forum.

Solution: Flip question order. When working with discussion forums, your first chance to engage your respondent may be your last and only chance, so get in with your best question first.

2. Challenge: Lower spontaneity. Forums and discussions are usually run over multiple days during which a participant is expected to log in multiple times and respond to the moderator’s posts and/or others’ posts. However, given the nature of the tool, the participants have a considerable amount of time to think about the question and post their response, with the ability to edit and adapt their post. We have noticed that discussion forums generally have a lot more people posting after they have processed their thoughts. Compared to a live chat, the spontaneity of the response is a lot lower.

Solution: Ask for detailed answers. Since participants are thinking more carefully about their responses, leverage that thoughtfulness by asking for detailed answers from the start…before they read others’ posts, or drop out.

3. Challenge: Uneven attention. Depending on how many you invite and how engaging the topic is, there are generally a lot of posts that get generated. Each participant’s post may be quickly followed by posts from the moderator or other participants, so unless participants log in often, they can easily miss a few posts. Most often, if participants have missed a day of the forum they will come in and start from the last post or just skim through things or just respond to the probes directly to them. Bottom line: with the volume of posts, not everyone reads everything in the discussion forum, which can leave participants with a feeling that no one is listening – which is the worst possible outcome, since it can lead them to disengage.

Solution: Moderator engagement. By acknowledging and probing participants’ posts, the moderator lets them know that their answers matter, and keeps them engaged in the conversation.

Applied to our example of a study on new tea flavours, here’s how our three key tactics would shape the research process:

1. Start by asking about the specific flavors the client is considering, so we are sure this key question gets addressed before participants drop out or reflect for too long. If the client is planning to launch two new flavors, you might ask participants about 6-10 prospective choices.

2. Post a question that encourages the participants to post their detailed view on more than a couple of these flavours, as opposed to picking one up and giving us some cryptic responses.

3. Probe with additional questions that ask the ‘why’ to develop your understanding of how participants feel about the flavoured tea category, and the landscape of tea perceptions. This style of probing and questions is quite similar to the deprivation or reverse laddering technique.

With this process, we have been able to collect the same buckets of information we would have found through offline interviews, simply by adjusting the order and type of questions to address the specific challenges of online forums. Best of all, we’ve been able to achieve those results without the time or costs of offline research.

Posted in: