Customer Experience

Prove it: 3 ideas on how to meaningfully measure CX transformation

Prove it: 3 ideas on how to meaningfully measure CX transformation

Everyone working in customer experience (CX) wants to know if their time and resources are well spent. CX can be a messy space, full of different beliefs and potential paths to follow. At the same time, CX has to compete with other departments for resources.

To earn credibility and influence in the organization and get more resources, CX departments need to prove their worth. This article looks at some examples of weaker and stronger evidence that CX professionals could consider.

Forget the grab-bag of “proof points”

“Ever since I joined the company, the share price has been trending up.” 

This is a kind of correlation that weaker business cases use. In general, we know that a claim like this one sounds flimsy, but businesses make these kinds of statements all the time. For example:

“During the two years since our program was rolled out, satisfaction scores improved by four percent; we also saw a five percent decrease in customer churn.”

The time-series-grab-bag is a place where many supposed proof points are generated. We all know that correlation does not mean causation, but when these examples are repeated often enough, they can sound compelling, and we start to believe them.

I understand that we don’t always operate in an environment where we can reject weak statements that claim to prove CX ROI. But accepting them without question can result in strategic wrong turns, and poor resource allocation. Modern medicine has been able to embrace a hierarchy of evidence that recognizes stronger standards of proof—why not business? But the good news for commercial organizations is that these principles have been spreading outside of scientific and academic research, and into our everyday business operations.

Just to be clear, I’m not suggesting CX professionals apply medical standards of rigor to every business decision. I want to point out that stronger evidence is out there for those important, business-transforming decisions that affect large sections of your customer base.

Let original ideas bloom with randomized controlled trials

CX programs typically involve design interventions, so when it matters, why not gather one of the most widely accepted forms of evidence: randomized controlled trials?

Randomized controlled trials are a hallmark in science and medicine, but they have spread to other sectors. When asked how the UK education sector could become more evidence-based, the number one recommendation from doctor and journalist Ben Goldacre’s was to promote a culture of randomized controlled trials. He provides this refreshingly simple outline of how to research “what works best”:

We split [participants] into two groups at random; we give one intervention to one group, and the other intervention to the other group; then we measure how each group is doing, to see if one intervention achieved its supposed outcome any better.

Goldacre thinks trials should be a routine part of life in education, but couldn’t that kill creativity? Actually, the experimental approach is also compatible with the innovation philosophy of the Lean Start-up. Within trials, highly original ideas can bloom as long as they are effective, where the same untested idea wouldn’t be as convincing.

Experiment over time with longitudinal research

If you’ve seen The Up Series, which interviews people every seven years, you’ll know that a longitudinal study can deliver much richer data than one-time surveys. A longitudinal study follows the same people over time and makes repeated observations. This approach is better able to link causes and effects, which is why it’s held in high regard by scientists in many fields.

For example, when a customer rates the brand highly, will they go on to spend more with the brand? This kind of question calls for a longitudinal approach.

Longitudinal studies can also help us develop better timing. When someone says they intend to switch brands, will the switch happen in a fortnight or be a lifelong pursuit? When a family moves house, what is the typical sequence of events, and when in that timeline can our business be helpful to them?

A large, well-designed study can be a bit like an always-on experiment to complement a transformation program. If the brand is fundamentally changing, let’s follow customers during that journey so that we won’t need to resort to a mostly correlational and usually useless grab-bag of data to document our success.

Final thoughts

This article has explored some extremes in evidence quality, leaving out plenty of good work in between. Here are three final reflections on what you can do as a CX professional:

  1. Protect your own credibility. Weak evidence may get you through some day-to-day decisions, but think twice about promoting it or publishing it outside of your organization.
  2. Consider the cultural impact of evidence-based innovation. Fairer resource allocations and meritocracy can be encouraged while providing a safe testing environment where original ideas can thrive.
  3. Have a conversation with your insight or strategy team about frameworks for gathering and sharing evidence into “what works.” It may take some planning, but it’s worth the time and effort.

The Enterprise Guide to Customer Experience (CX)



Subscribe to the Vision Critical blog

Get free customer intelligence tips and resources delivered weekly to your inbox.

By completing this form you consent to receive emails from Vision Critical. You can unsubscribe at any time. Learn more in our privacy policy.