Wondering how to determine what's most important when it comes to your offering?
Ranking and Rating Dimensions Won't Work
To make decisions about a brand or product, we often want to ask a group of people what dimensions are important to them. For example, which features they want in a new app, or which components of an experience that would make members feel special.
The typical approach to do this was to either ask consumers to rate or rank the dimensions.
- Rating (i.e., a five-point scale from “not at all important” to “extremely important”) often resulted in ratings clustered together at the high end of the scale.
- Ranking brings about greater discrimination (no two dimensions can be tied), but is also a difficult, time consuming, cognitive task for respondents.
MaxDiff Is Cumbersome and Artificial
Maximum Difference Scaling (MaxDiff) was introduced as an alternative, attempting to solve the rating and ranking problems. Rather than have each respondent rate or rank all the attributes at once, MaxDiff presents smaller subsets to respondents. Of four or five dimensions, a respondent is asked which one is most and least important. And the process continues several times with other subsets. The partial rankings are then accumulated and serve as the basis for statistical analysis and reporting, with a one-number summary (utility) provided for each benefit.
MaxDiff is not a bad technique, however, it is a bit over-engineered with a cumbersome respondent experience to capture relatively simple information. While meant to enhance sensitivity and ease cognitive strain by presenting benefits in smaller sets, MaxDiff does not match the way consumers think about ordering importance or mirror a real-life decision-making process.
Further, we have found via research on research that a rating approach, regardless of scale, results in comparable rank orders of attributes to MaxDiff—both approaches find the same attributes as most and least important.
Try a Simple Approach That Matches Thought Processes
What if there was a simpler way that still had the benefits of MaxDiff? Selection is a measurement process based solidly on heuristics that asks respondents to first read through the list of options to be evaluated and then to select just those that are of greatest relevance to them in their decision-making process (i.e. a multi-choice question). The percentage selecting each benefit is the relevant summary statistic. This survey process is even simpler than ratings, with even less cognitive strain, and matches most closely the actual thought process. By not using scale ratings, this method removes concerns with scale bias, especially related to cultural scale issues.
Next time you or someone in your organization needs to determine what dimensions are most important to your audience, steer away from MaxDiff. Instead, ask people to simply select what’s important to them in a multi-choice question. You’ll treat your audience or members with respect and have results that you can trust to make a decision.