Is Net Promoter Score flawed?
Jeff Berry, Editor-in-Chief of COLLOQUY and Senior Director of Research and Development at LoyaltyOne
Through a special arrangement, what follows is a summary of a special report from COLLOQUY, provider of loyalty-marketing publishing, education and research since 1990.
For more than a decade, the yardstick for measuring business success has been Net Promoter Score (NPS), a tool created to gauge customer satisfaction with a single question: “How likely is it that you would recommend our company/products/services to a friend or colleague?”
Answering on a scale from zero to 10, those giving a score of nine or 10 classified as “promoters” of the company, product or service. Seven or eight scorers are considered “passive”; their responses are essentially thrown out in the final calculation. Zero through six are rated “detractors.” Subtracting the detractors from the promoters yields your Net Promoter Score. Any score above zero is generally considered positive and anything above 50 is excellent.
Yet NPS has also been the subject of criticism and controversy from academics, market researchers and business analysts who question whether it actually moves the needle on consumer behavior and corporate profitability. As with most things in life, Net Promoter Score is probably neither as exalted as proponents claim nor as flawed as critics do.
Strengths of NPS are seen as its simplicity (ease of use/understand), familiarity, benchmarking relativity (many brands use it) and adaptability across industries. One weakness is that NPS is a rear-facing metric: consumers’ attitudes are generally informed by their most recent experience. The response scale can also be confusing or culturally biased and the question has an inherent positive bias.
One inherent flaw is its say-do gap. Our COLLOQUY research found that only 13 percent of customers actually go on to make a recommendation after answering that they would. B2B research also finds only five percent of companies follow up with consumers who were asked the NPS question and ask them to make that recommendation.
In order to refine NPS to be a better predictor of consumer behavior, companies must ask pointed follow-ups: “When will you make this recommendation? Let’s do this!”
Companies must also push to discover the “why” or “why not” of the consumer’s decision to make the recommendation. This can provide an opportunity to change consumer behavior in a way that’s meaningful, rather than just measuring it. If you want to get a real research answer, you need to get at customers’ underlying intent, motivation and belief systems in ways that are not as simple.
Discussion Questions
DISCUSSION QUESTIONS: Do you see more of an upside or downside to Net Promoter Score as a measurement tool? What adjustments or extra steps need to be made to improve its effectiveness?
Poll
BrainTrust
Doug Garnett
President, Protonik
Adrian Weidmann
Managing Director, StoreStream Metrics, LLC
I’m a fan of NPS, I know what it is good for — and what it is not good for. It is a one-question survey that gives you an instant read on a customer’s overall feeling for the company. Being willing to recommend is different than actually recommending. But the fact that a customer is happy enough to possibly do so tells you a lot about how the customer feels about their experience. Adding an additional question or two — maybe an open ended question, such as “Why did you give us that score?” — may be helpful. I’ve written quite a few articles on NPS and I don’t want this answer to turn into another one. So recognize NPS for what it is — a simple, yet powerful, question that gives you a good sense of what customers think of what it’s like to do business with you.
I think Net Promoter Score is an adequate measuring tool. However, that can’t be the only question retailers are asking customers in the survey. There needs to be other questions on sales associates, quality of the product, the atmosphere of the store, etc. If someone ranks an element low, there should be an opportunity to explain why. There are several customer experience platforms that give survey takers the ability to write a recommendation to be shared on social media when they rate a company as a “9 or 10/would recommend.” The problem isn’t necessarily with NPS, it’s with how retailers are collecting the data and what they’re doing with it. Using an advanced CX platform with additional capabilities is a must.
Theories like the NPS need to be opposed at a fundamental level. By taking an incredibly subtle and complicated real world and simplifying it beyond belief they lead marketers into incredible error. (Campbell’s Law should be enough to scare us off of paying much attention to things like NPS.)
Measures like NPS are seductive — helping create simple reports that everyone can understand. But market health is NOT simple and not everyone should be able to understand it. And executives need to resist the politics of adopting false simplicity.
The clincher for me, though, is that Byron Sharp (author of “How Brands Grow”) has particularly attacked NPS for not being a reliable predictor of future growth. Here’s a link to one post by Sharp about NPS.
Everything I’ve read says it’s a weak predictor of future behavior. But then, a lot of those studies have assumed that a promoter actually has to promote to be counted as following through. I think that requirement is what hurts. We like lots of products and businesses but it doesn’t mean we’re going to spend time actively promoting them. It’s a good, simple measure of whether you are delivering on expectations — nothing more.
My inability to grasp the subtle nuances of marketing trends du jour is well documented so forgive me if I demonstrate it once again. I always thought the NPS question was flawed. Why ask if, at some indeterminate point in the future, one might recommend something to a friend or colleague — assuming that colleagues aren’t friends, I guess — when the right question, (assuming there is a right question,) ought to be something more like, “Have you ever recommended our company/products/services to a friend or colleague?”
Anyone can say they might recommend a company/product/service, but isn’t it more important to understand whether or not they actually do it? And when you think about it logically, isn’t recommendation really a binary choice — either you would or would not recommend? How does an unqualified gradient help you understand behavior better? In my business I don’t really want to hear that my clients might recommend me to someone else. I much prefer hearing that they have. You can’t bank on intentionality.
I don’t believe it is flawed at all. It gives you a very quick indicator of how your business is performing at that point in time. It is also simple for the customer to grasp and to answer.
Let’s face it, in our marketing quest to get information on the customer, we intrude with long drawn out surveys, which are totally obsolete given the tools we have today.
We use it constantly in auto-responder email sets after the customer has gotten their purchase and has had the time to use the product. If they do not feel comfortable recommending your product, then you need to delve deeper at that point.
We know that referrals drive other purchases. We understand the power of referrals. So why would a score based upon customers referrals be outdated?
NPS, while simple and providing a way to benchmark against industry, isn’t enough by itself. Simply knowing that someone would recommend an organization (and some types of organizations lend themselves to this concept better than others) doesn’t, by itself, provide any clear direction about what is going well and what needs to improve. One might be willing to recommend a company but there could still be issues in one facet (or more) of the experience. Alone, NPS is simply not enough.
Depending upon responses to a question about how likely someone is to recommend a product is highly overrated. It is easy to answer the question positively. However, does the product ever come up in conversation? Do I introduce it into the conversation? How credible is my recommendation? Do others act on my recommendation? Relying on the answer to the first question without knowing the answers to the rest of the questions is not a wise move.
First, I’m not a huge fan of Net Promoter — but there’s nothing inherently “wrong” with it either.
The measure simply has to be taken in context. It is not a good predictor of future behavior if your standard is “how many of those people will actually go on to recommend me.” But it is a pretty clever surrogate for sussing out the true level of a customer’s satisfaction with their experience(s) to date.
The one technical bone I would pick (unfortunately forcing me to sort of agree with Ryan) is that the question should be binary — YES or NO — introducing the nuance of a Likert scale is contrary to the notion of getting a simple answer to a simple question. And while I gotta believe they have some, I have never seen any statistical justification for the breaks between 1-6; 7-8 and 9-10.
NPS has some real strengths — it is easily understood and can provide some good diagnostics when used well. A couple of caveats:
Total NPS trends are not as valuable as taking action and learning from the individual scores.
“How likely is it that you would recommend our product?” isn’t nearly as important as “How does our product make you feel about yourself?” While NPS does have value in guiding marketing decisions, relying on solely the NPS to manage the operational levers that drive decisions is ignoring the broader context of what really makes consumers happy. NPS ignores the emotional context while focusing on a single metric. Adding in emotional context to NPS information greatly multiplies the impact of those levers.