Souce: Netflix.com
Should retailers ditch five-star ratings?
Five-star online ratings for reviews are common across retail (Amazon, Macy’s, Walmart, etc.), but Netflix believes the system is too complicated and biased.
The video streaming service is swapping its star system for a binary “thumbs up, thumbs down” one. The ratings are used to make personalized recommendations.
Netflix said a “thumbs up, thumbs down” test with hundreds of thousands of members in 2016 led to 200 percent more ratings than the traditional star-rating feature. Todd Yellin, Netflix’s VP of product, said at a press briefing in mid-March, “We are addicted to the methodology of A/B testing.”
Beyond gaining less feedback, the star system was found to be a poor method of discerning which shows and movies people liked.
Behaving like critics, users tended to rate the quality of a film rather than basing it on their enjoyment. For instance, documentaries would often receive five stars and low-brow comedies three stars, but users watched comedies much more often.
“What’s more powerful: you telling me you would give five stars to the documentary about unrest in the Ukraine; that you’d give three stars to the latest Adam Sandler movie; or that you’d watch the Adam Sandler movie 10 times more frequently?” said Mr. Yellin.
YouTube switched from a five-stars to a like/dislike system in 2010 citing that users had overwhelmingly been rating videos five stars. The next most common was one star. Two, three and four were rarely chosen.
Retailers have tweaked their ratings systems over the years. In 2015, Amazon introduced a new machine-learning platform that gave more weight in star ratings to newer reviews, those from verified purchasers and reviews voted as being helpful.
Lately, a push is being made to make ratings more informative. A Wall Street Journal article from December noted how Target has added descriptors so reviewers can award stars in categories such as “ease of assembly” for furniture or “design” for home decor items.
On Macy’s website, reviewers often provide their age range, gender, location, how frequently they purchase at Macy’s, a description of their style and the occasion they bought the product for.
Discussion Questions
DISCUSSION QUESTIONS: Is the five-star ratings system too flawed to be used by retailers as a recommendation engine? What ratings methods do you think work best in producing helpful advice for online shoppers?
Poll
BrainTrust
Shawn Harris
Board Advisor, Light Line Delivery
Jasmine Glasheen
Content Marketing Manager, Surefront
I believe technology will not only save the five-star system, but improve it. With machine learning and auto-content creation, the technology exists to scan a large volume of reviews in milliseconds and then auto-generate very focused, digested articles with no human writers involved. These articles can, for instance, list the top 10 reasons why reviewers gave a product five stars and the top 10 reasons why reviewers cited problems with the product.
Ratings are used to get feedback from customers as well as to share that feedback with the public. Consumers use the ratings to make decisions. The flaw in the five-star rating system will be the same in a thumbs-up/down system. One is just a little “simpler” than the other. For example, someone who can’t get a reservation to a busy restaurant is upset and gives the restaurant a poor rating (one star or thumbs down) — yet the customer never even stepped foot in the restaurant. The reasons ratings work is because of the average of a number of ratings, not just one single rating. I prefer the five-star rating, but either works as long as there is a place for editorial comments from customers.
For a company like Netflix a simple thumbs up or down rating system may work — Siskel and Ebert used it for years. For companies like Amazon consumers may prefer a more detailed five-star rating system. One system does not need to fit all retailers.
Similar to Pandora with music, Netflix is looking to solve a yes/no question — “should we recommend more movies like this one?” so a binary rating makes perfect sense for them.
Most retailers also want to know more complicated questions such as: “How did customers enjoy shopping at my store compared to their experience at a competitor’s store?” Or “Is the shopper’s experience at my store improving over time?” A scaled rating system (like the five-star rating system) is a much better way to answer those questions.
I believe that the social proof the five-star rating provides is still critical for retailers. There is credence in the ratings, which undoubtedly leads to increased conversion rates and weeds out poorer performing products.
Reviews have their place, certainly. But it depends on the application. On marketplaces like Amazon the reviews are more credible (since Amazon clamped down on fake reviews) because of the purchase requirement. Consumers do not gravitate towards the good reviews, they look at the bad reviews to see what the complaints with the product are and how the company responded. So in that respect, good and bad reviews can be a powerful brand tool.
With consumer products, online reviews on third-party sites carry more credibility to a consumer. Reviews on a brand’s site, gathered by the brand, will always be viewed as suspect.
Movie reviews are different and need more clarification. I don’t believe stars are the right methodology to use. I think words and explanations are more powerful tools. More so than just thumbs up or down.
I believe the effectiveness of the rating system used (five-star or thumb up/down) depends on what type of merchandise or service is being rated. Even more important than the rating is the commentary offered by reviewers. I expect a better system will be developed using machine learning/AI systems that will process the written commentary and calculate an unbiased rating based on those comments rather than allowing the individual reviewers to assign the rating. This implies a meaningful number of products must be sold to generate the rating or a service needs to be used by enough customers to be significant. In many cases I’d say having multiple ratings for defined characteristics is important vs. just having a single overall rating.
Both five-star ratings and the thumbs-up/thumbs-down approach are too simplistic to address the multiple facets of omnichannel retail. A web-generated bar graph showing popular consumer responses would be more effective and resolve rating discrepancies between quality and accessibility.
Like it or not, the five-star rating system works quite well, especially when there are significant enough responses to provide a range. Customers leverage the five-star rating system to base their initial decision making on online products, video streaming, etc. As we are in the age of digital natives, beyond the simple five-star ratings, you can take it a step further and read customer and professional reviews to determine if this is the right product for you. This is the age of customer empowerment and, as we know, knowledge = power.
While the five-star system is still evolving, diluting it back down to a thumbs-up/thumbs-down strategy is a step backward.
Thumbs up or thumbs down works for video. Stars work for products. I don’t understand the premise of this article.
Wow. We have officially become a zero-attention-span / instant-gratification / I-have-no-time-to-think-about-anything society. If it isn’t “yes” or “no,” then I can’t give any more specific thoughts on the topic, sorry. Seriously? Wow.
With something like Netflix, the main question a rating should answer is whether I should watch something. With retail products, there are a wider variety of variables that requires a gray area in terms of ratings. Did a function not perform as advertised? Were your expectations only partially met? Providing ratings for digital content versus physical merchandise are two separate beasts. A more complicated rating system would be more effective for consumers to gauge whether they should add to cart, but then there’s the issue of whether people would be willing to spend the time to leave a full review.
Ratings are here to stay. Netflix can do what they want, but the rest of the online world has been using 5 star ratings for years. There is no magic switch to turn them off or on, or push consumers to only use your “system.”
“Netflix believes the system is too complicated.” The dumbing-down of America continues.