Vision Critical

The customer insight platform
for customer-centric companies

Get a Demo

Wondering how to determine what's most important when it comes to your offering?

Ranking and Rating Dimensions Won't Work

To make decisions about a brand or product, we often want to ask a group of people what dimensions are important to them. For example, which features they want in a new app, or which components of an experience that would make members feel special. 

The typical approach to do this was to either ask consumers to rate or rank the dimensions.  

  • Rating (i.e., a five-point scale from “not at all important” to “extremely important”) often resulted in ratings clustered together at the high end of the scale. 
  • Ranking brings about greater discrimination (no two dimensions can be tied), but is also a difficult, time consuming, cognitive task for respondents.

MaxDiff Is Cumbersome and Artificial

Maximum Difference Scaling (MaxDiff) was introduced as an alternative, attempting to solve the rating and ranking problems. Rather than have each respondent rate or rank all the attributes at once, MaxDiff presents smaller subsets to respondents. Of four or five dimensions, a respondent is asked which one is most and least important. And the process continues several times with other subsets. The partial rankings are then accumulated and serve as the basis for statistical analysis and reporting, with a one-number summary (utility) provided for each benefit.

MaxDiff is not a bad technique, however, it is a bit over-engineered with a cumbersome respondent experience to capture relatively simple information. While meant to enhance sensitivity and ease cognitive strain by presenting benefits in smaller sets, MaxDiff does not match the way consumers think about ordering importance or mirror a real-life decision-making process.

Further, we have found via research on research that a rating approach, regardless of scale, results in comparable rank orders of attributes to MaxDiff—both approaches find the same attributes as most and least important.

Try a Simple Approach That Matches Thought Processes

What if there was a simpler way that still had the benefits of MaxDiff? Selection is a measurement process based solidly on heuristics that asks respondents to first read through the list of options to be evaluated and then to select just those that are of greatest relevance to them in their decision-making process (i.e. a multi-choice question). The percentage selecting each benefit is the relevant summary statistic. This survey process is even simpler than ratings, with even less cognitive strain, and matches most closely the actual thought process. By not using scale ratings, this method removes concerns with scale bias, especially related to cultural scale issues.

Next time you or someone in your organization needs to determine what dimensions are most important to your audience, steer away from MaxDiff. Instead, ask people to simply select what’s important to them in a multi-choice question. You’ll treat your audience or members with respect and have results that you can trust to make a decision. 

Insight-Driven Decision Making

Text Size

- +

Robert Hein

Robert helps Vision Critical’s customer-facing teams deliver more value to customers at every possible touchpoint in their journey. Managing the Sparq Next insight community, he connects the customer’s voice to the product, marketing, and customer success functions. His practical applications of insights supports the customer-centric mission of Vision Critical.

He designs, measures, and improves customer experiences with a solid connection to our business goals and history. When he’s not championing the customer journey, you can find Robert methodically creating ikebana, Japanese flower arrangements.
Say hello: