How Do You Measure Customer Satisfaction?

The Net Promoter Score (NPS) is a metric that is widely used among retailers to measure their level of customer satisfaction, determine approaches to improve that satisfaction, and measure their performance against competitors.

NPS, developed by Fred Reichheld, was introduced in a Harvard Business Review article in 2003 and received broad acceptance through his 2006 book, The Ultimate Question. NPS allows companies to evaluate customer satisfaction based on a single question: "On a 0 to 10 scale, how likely is it that you would recommend us to a friend or colleague?"

Mr. Reichheld divided customer responses into three categories: Promoters (those who respond with a nine or 10 score), Passives (customers with scores of seven or eight), and Detractors (customers with scores of six or below). The NPS is calculated as the percentage of promoters minus the percentage of detractors. Companies often do further research on detractors in order to improve the customer experience and increase overall NPS.

At the recent eMetrics conference in Boston, Foresee, a company that specializes in measuring online customer experience, introduced an alternative metric to NPS called the Word of Mouth Index (WOMI). WOMI specifically identifies customers who will go out of their way to promote a brand and compare those to customers who will go out of their way to discourage use of a brand.

ForeSee tested and refined WoMI over a two-year period with more than 300 companies and through nearly 1.5 million customer surveys. WoMI enhances the NPS "How likely are you to recommend?" approach with a second question: "How likely are you to discourage others from doing business with this company?" WoMI then subtracts the percentage of nine and 10 ratings from "likelihood to discourage" from the percentage of nine and 10 ratings from "likelihood to recommend" to arrive at the balanced and accurate WoMI score.

The WOMI formula is calculated below:

womi cht

The word-of-mouth index presents an opportunity for retailers to improve the validity of NPS and at the same time challenges them to revisit an already accepted metric. If a company wishes to continue using NPS, it can improve on NPS simply by adding WoMI’s second "How likely are you to discourage?" question to get a more direct measure of the presence of consumers who actively discourage the brand — those consumers who need direct attention and need it right away.

BrainTrust

Discussion Questions

What metric(s) are most valuable to measure when it comes to customer satisfaction? How valuable will WOMI be to retailers seeking to understand customer satisfaction and take action to improve performance?

Poll

17 Comments
Oldest
Newest Most Voted
Inline Feedbacks
View all comments
Ken Lonyai
Ken Lonyai
10 years ago

Both NPS and WoMI are outside-in measurements, meaning: they are survey questions asking opinions of customers looking back at an experience. Of the two, I like WoMI better, but the outside-in approach is always flawed.

Most 3rd party vendors are not given access to sales/user/loyalty data so they have no choice but to get user opinion. While user opinion can be accurate, it does not always reflect true thinking and habits. So, internal measures of loyalty and return sales will always provide greater depth and meaning to user experience/happiness.

Richard J. George, Ph.D.
Richard J. George, Ph.D.
10 years ago

Regardless of the specific metric chosen to measure customer satisfaction, one cannot underscore the power of positive and negative word of mouth. Chick-fil-A generates 75% of its new customers via word of mouth. Obviously, the Chick-fil-A experience is creating a significant amount of positive word of mouth or true promoters.

Social media has become the “digital backyard fence” allowing individuals to share their good and bad customer experiences with their family, friends and the connected community. How one measures word of mouth is less important than whether it is measured and then responded to in order to minimize discouraging customer comments.

Bill Davis
Bill Davis
10 years ago

While both measures are useful, I personally would prefer WOMI because it offers the extra question. That being said, I don’t believe customer satisfaction can truly be measured in a single question or two.

The world isn’t so black and white – you’re either with us or against us – and neither of these measures really does a fantastic job of measuring the gray areas where a customer is both pleased/displeased with a retailer. Like any normal distribution, the majority of customers will fall in the middle and not at an extreme.

That being said, the extremes most definitely need to be heard and managed as their impact can be greater than their numbers.

James Tenser
James Tenser
10 years ago

For true believers in the quality-satisfaction-loyalty linkage, metrics like NPS and WOMI bring apparent rigor to the analysis.

But I’d urge a healthy skepticism in interpreting any self-reported measures like these unless you can draw clear linkages to business performance metrics.

Satisfaction itself breaks down to a number of dissimilar component influences, ranging from must-haves (like hours of operation, items in-stock, and decent pricing) to delighters (like bonus deals, personalized services, and free samples). In between are foundational elements, like convenience, location, competitive alternatives, and a desirable assortment.

So as a summary construct, “How satisfied are you on a 1-9 scale?” rings rather hollow for me. “How likely are you to recommend/discourage…?” is only a little better. Both lack what I would call diagnostic value. It takes deeper and finer inquiries than those to root out opportunities and define beneficial actions.

Raymond D. Jones
Raymond D. Jones
10 years ago

We have used a variety of metrics in our customer satisfaction or voice of the customer work. Provided they have validity and are used consistently, they can be very useful. However, one must go further for them to be really actionable. You need to understand the reason for the customer rating and what steps it requires.

Ideally, the attitudinal metrics need to be confirmed by customer behavior. Loyalty or repeat purchase is a more powerful measure than a positive intention or recommendation.

Chris Petersen, PhD
Chris Petersen, PhD
10 years ago

I agree with Ken Lonyai’s assessment that the challenge lies with “outside-in” methodology. Both NPS and WOMI are “after the fact” surveys of consumer perceptions of the experience, which can be influenced by a number of factors.

Case in point, if you have purchased an automobile recently, you quickly learn how customer satisfaction metrics can be subverted when they used for other purposes like personnel evaluation. I can’t begin to count the number of times the dealer, salesman, and service department said “if you can’t rate us perfect 10s, then tell us so we can make it right, so that you can rate us a 10.”

In this age of social media, survey scores like NPS and WOMI need to be balanced with some indices of what consumers are actually communicating to their friends and networks. There is some great work emerging on analyzing consumer comments and tweets that can provide both balance in metrics, and deeper insights into consumer satisfaction and loyalty.

Ronald Stack
Ronald Stack
10 years ago

I agree with most of the previous comments, so I will only add this: We are asking for emotional responses, and the intensity of emotions changes over time.

Ask an “intend to recommend” question right after a customer has gotten an unexpected bonus and the score may be very high. Ask the same question of the same shopper six weeks later and the intensity (but probably not the valence) is likely to be different.

Ask the same question of a shopper who has just received poor service and the negative response may be strong. Take immediate corrective action and not only will the intensity likely be less, the valence may even flip. Ignore that same customer for six weeks and the negative intensity might be off the scale.

In short, I think it’s important to control for when in the interaction these questions are asked.

Lee Kent
Lee Kent
10 years ago

Everybody needs a yard stick and this is the best thing going. I would just throw in one more thing. With either of these metrics, I would want to know what the influence level of the respondent is too. Just sayin’.

Arun Channakrishnaiah
Arun Channakrishnaiah
10 years ago

I think both these indices place more (than warranted) importance to the outliers (the customers who actively promote/discourage). It probably is more beneficial to understand the set of customers who fall outside of these brackets. I’m referring to the customers who either did not participate in the surveys or gave average scores. Relying on these kind of metrics alone (repeat alone) will mean that the noise of a few drowns out the voice of the majority.

Jonathan Marek
Jonathan Marek
10 years ago

James Tenser hits the nail on the head on this one. What you really need are diagnostics that help you go in and fix the problems, and this gets you just a little way down the path. So ultimately, I’d say either measure is fine and fine-tuning this measure further is of dubious value.

Jason Goldberg
Jason Goldberg
10 years ago

WoMI is technically a better implemented metric than NPS…(more granular, less data quantization error, etc…). But part of the benefit of these external metrics is to be able to benchmark vs. others, and obviously NPS is MUCH more broadly used than WoMI.

NPS is a “good enough” metric for which a huge amount of benchmarking data is available. I was interested to hear at shop.org about how much Tory Burch has been able to leverage NPS to improve their business.

Marianne Klinker
Marianne Klinker
10 years ago

When it comes to measuring satisfaction, WoMI is a great next step for companies already using NPS, however, it is essential to include it in a broader set of analytics that can provide a true measurement of the customer experience.

Also, for those looking for WoMI Benchmarking data, a variety of industry benchmarks are available in Larry Freed’s new book, “Innovating Analytics.”

In addition, ForeSee releases quarterly WoMI benchmarks for the top 100 brands. Today we released the Fall 2013 edition that provides benchmark scores for satisfaction, the Word of Mouth Index (WoMI) and Net Promoter Score (NPS) for 100 of the top brands in the world and finds NPS overstates detractors for the biggest brands by 780% on average.

Ralph Jacobson
Ralph Jacobson
10 years ago

Along with the metrics mentioned here so far, it is also very possible to literally create your own metrics, and compare those year-to-year, along with your industry peers performance with those metrics. You can create a metric to measure performance in a key area of your organization.

You can create at least three types of metrics. One, traditional metrics for which values are loaded (like the ones mentioned here in the article), two, derived indexes which use scores of other metrics to generate a status and a score and is useful for tracking the performance of related metrics, providing a numerical assessment of something that cannot be directly measured, and generating an overall status and score of multiple metrics.

When you define a derived index, you also include rollup rules. You specify rules to set the status and score of the metric. For instance, a “weighted average” rollup could be based on the scores and the weights of the impacting metrics. For each impacting metric, multiply its score by its weight, add the results of each calculation, and divide that number by the total weight. Example: Four impacting metrics, Returns, Complaints, Customer Survey, and Repeat Customer Percentage create a calculated metric named Customer Satisfaction Index. The score, status, and weight for each metric are as follows:
>> Returns 5, green, 10
>> Complaints 0, green, 5
>> Customer Survey -10, red, 20
>> Repeat Customer Percentage -1, yellow, 15

The calculation would be ((5*10)+(0*5)+(-10*20)+(-1*15))/50 = -165/50 = -3.3.

Therefore, the Customer Satisfaction Index is -3.3 and the status is red. The color depends on where the score falls in relation to the default tolerances.

Finally, the third type of metric is “Formula-based calculated metrics” that use mathematical operations to generate a value for actual, target, tolerance, or user-defined criteria.

I think we have to be a bit more creative with our metrics to glean more insights from the consumer behavior.

Phil Rubin
Phil Rubin
10 years ago

Reicheld developed NPS as a metric that was better than customer satisfaction (CSAT) and more focused on customer loyalty. However even Reicheld himself has backpedaled from his “One Number” stance, largely based on the fact that NPS, like CSAT, isn’t a useful predictor of customer behavior. Measuring whether someone “would” recommend is different than measuring whether someone “will” recommend.

In our work, the “ultimate” metric for measuring customer loyalty is CompCustomer sales – whether the (exact) same customers are buying more now than they did prior. More on this can be found here.

Mike Osorio
Mike Osorio
10 years ago

It is a question of balance. Measures like WOMI, NPS, Gallup CE13, etc, are all good measures that offer leaders a consistent view of customer sentiment over time. Today’s technology also allows us to begin mining social media to get an immediate and rich base of sentiment analytics as our customers “talk” about us on their various social platforms. Add to this traditional tools such as mystery shopping and employee engagement surveys, and leaders have a full picture to develop ongoing action plans to move the customer engagement needle ever more positively.

Shep Hyken
Shep Hyken
10 years ago

The NPS is a great way to get a 30,000 foot view of the customer’s perception of your company. It is a powerful measurement that gives you a snap shot of the customer’s overall experience – good enough (or not) to recommend you. What it doesn’t do is get you specific details. But, that doesn’t matter when it comes to WOM. Either the customer will or won’t promote you.

WOM is the most powerful form of promotion and advertising. The message to do business with the company is coming from a trusted resource; friends, colleagues and family members. It doesn’t get better than that!

Bryan Pearson
Bryan Pearson
10 years ago

Both measures offer a simple approach to understanding current customer sentiment and by their widespread adoption (more NPS than WOMI) a relative comparison to other brands, either within or outside the category.

However, the failing in these measures is the ability to put into action any of the insights that might be gleaned from understanding your ‘score’. Not unlike customer satisfaction studies, I think these measures are largely intended to act as a scorecard and inform annual performance bonuses.

Where the rubber meets the road is in the ability to both measure and act. At COLLOQUY we have been measuring what we call Word-of-Mouth Champions, those customers that claimed a willingness to recommend and actually made the recommendation, as a far greater predictor of customer value. By closing the say-do gap through the triangulation of stated and observed behavior, not only are you able to target offers with better precision but you are able to identify higher long-term value customers that can actually be engaged to better understand the true drives of customer advocacy – functional satisfaction and both habitual and emotional drivers of engagement while managing the recency effect that is inherent in NPS and WOMI.

The COLLOQUY Word of Mouth research is posted here.