The belief that online user ratings are good indicators of product quality is largely an illusion, according to a new University of Colorado Boulder study.
Researchers examined user ratings for 1,272 products across 120 product categories, such as car seats, bike helmets, sunblock, air filters, smoke alarms and blood pressure monitors. Their analyses showed a very low correspondence between average user ratings of products on Amazon.com and product ratings, based on objective tests, found in consumer reports.
“The likelihood that an item with a higher user rating performs objectively better than an item with a lower user rating is only 57 percent,” said Bart de Langhe, author of the study and professor of marketing at CU-Boulder’s Leeds School of Business, in a statement. “A correspondence of 50 percent would be random, so user ratings provide very little insight about objective product performance.”
The researchers didn’t investigate the reasons reviews may be a poor indicator of product quality. However, they did point out that when reviewers comparing two products with the same objective qualities, they tend to give higher ratings to the one that is more expensive or from a brand with a premium reputation.
An often-heard criticism of online reviews is that a “herd mentality” leads those who read positive reviews to be more likely to leave positive remarks themselves. Social media also feeds this herding effect. Wrote Sinan Aral, the David Austin Professor of Management at the MIT Sloan School of Management, in a white paper, “We tend to herd on positive opinions and remain skeptical of the negative ones.”
- Consumers’ trust in online user ratings misplaced, says CU-Boulder study – University of Colorado Boulder
- Navigating by the Stars: Investigating the Actual and Perceived Validity of Online User Ratings – University of Colorado Boulder
- The Problem With Online Ratings – MIT Sloan Management Review
- Can You Trust Online Reviews? – Kiplinger