Photo: Canva
August 15, 2023
Are Incentivized Reviews Still Biased Even With Full Disclosure?
The Federal Trade Commission (FTC) recently implemented guidance that reviewers must fully disclose any incentives they receive in exchange for reviews. A new study finds the guidance doesn’t go far enough.
The study, from University of Florida and University of South Carolina researchers, shows that when people received free products in exchange for reviews, their ratings were significantly inflated — and product sales were stronger — even though reviewers disclosed that they received the product for free.
The research was based on thousands of reviews posted on Amazon before and after October 2016, when the platform stopped working directly with reviewers to offer free products in exchange for a review.
Incentivized reviewers were found to rate identical products nearly 0.5 stars higher on average on a five-star scale. Researchers found that many reviewers felt compelled to post “overwhelmingly positive reviews” so the retailer continued to hire them.
Amazon’s removal of incentivized reviews on the same products led to a significant sales reduction, but customer satisfaction increased, marked by a smaller number of one-star reviews.
Researchers also found that marketplace-led review programs — such as Amazon Vine, in which Amazon selects reviewers rather than the seller — support more unbiased reviews.
Woochoel Shin, a University of Florida business professor, said, “If you’re Amazon, you’re not interested in inflating the rating of one specific seller. Amazon has an incentive to keep the marketplace fair, but they also want to provide information to consumers. Reviewers, knowing that the party that hired them does not want to inflate the rating, offer fairer reviews.”
The researchers noted that incentivized reviews remain common online, including on sites such as Walmart, Yelp, and Overstock. They also note how influencers routinely hype products in paid reviews on Instagram and YouTube.
Encouraging reviews is often recommended for businesses seeking to offset the challenges of removing negative fake reviews from a competitor or disgruntled ex-employee.
BrightLocal’s Local Consumer Review Survey 2023 found that despite platforms such as Yelp and Google also moving toward stricter policies against solicitation in reviews, offering discounts or cash in exchange for reviews saw a rebound in 2022. When asked which ways survey responders have been incentivized to leave a review by a local business in the last 12 months, the top responses were in exchange for a discount (26%), for a gift or service (18%), and for entry into a competition or prize draw (17%).
Discussion Questions
DISCUSSION QUESTIONS: Does requiring full disclosure around incentives do much to reduce the number of inflated online product reviews? In what ways should retailers encourage reviews, if at all?
Poll
Loading …BrainTrust
John Lietsch
CEO/Founder, Align Business Consulting
Gary Sankary
Retail Industry Strategy, Esri
Jeff Sward
Founding Partner, Merchandising Metrics
Recent Discussions
More Discussions
Retail Experts on Holiday 2025: Consumer and Retailer Confusion, and AI Expectations
Retail experts suggested private label is here to stay, AI is no mere hype machine, and…Are Restaurants Shrinking the Thanksgiving Opportunity for Grocers?
Restaurants could be stealing some valuable spend from grocers traditionally affiliated with Thanksgiving dinner. Do you…Would Recording In-Store Associates Help as a Training Tool?
Ethosphere is banking on AI-driven recordings of in-store associate practices as the next big thing in…
Yes, an incentivized review is biased – even if the bias is unconscious. This is especially so where someone receives a discounted or free product. That upends any conception of the value equation. It likely also makes someone think more favorably about a brand. All that said, most reviews are very subjective as one person’s definition of taste, quality, design, etc. is not the same as someone else’s. Caveat emptor always applies when buying things!
Incentified reviews should not be commingled with organic reviews, as it distorts the true averages of star ratings, IMHO. If retailers are offering incentives for reviews, in the ideal world, there should be two categories of reviews. I suggest the category of organic reviews should be the true average for star ratings, as they are unbiased. Reviews that are based on incentives, should be separate and properly identified and not include star ratings.
On the money!
It is human nature to experience a positive subconscious reaction if one is being rewarded for one’s opinion.
Of course a review written with incentives in place is biased. Human nature..why would anyone write a bad review when there are bonuses for writing a good one???
The term “incentivized review” is a pure oxymoron. I recently pulled off the freeway and stopped at a McDonald’s for a quick bite and a cup of coffee. For some reason I actually read my receipt. It said, “Rate us 5 (Highly Satisfied) and receive a free order of medium fries or a bakery item.” That’s not a review. That’s a bribe. And has zero role in being included in any kind of “review” process. Utter nonsense. Any incentives render the word “review” to be utterly meaningless.
I love oxymorons.
Hilarious! But please send me your receipt; I’m addicted to McFries! By the way, my other favorite is the “guilt trip close” after customer service calls. “You will receive a survey after this call. I know you couldn’t understand me and it took me 37 minutes to figure out your problem and another 20 to ‘not really solve it’ but please make sure you rate us a “5” so I don’t lose my job!”
I tossed the receipt, but it’s memorialized forever in a LinkedIn article. Here’s a link. https://www.linkedin.com/in/jeff-sward-3769251/recent-activity/all/ Enjoy the fries!
Is a paid influencer an influencer or a spokesperson? When you are paid to review then your primary objective becomes “getting paid” and not necessarily providing unbiased, valuable information. I think requiring full disclosure of reviewers who are paid helps consumers value that review accordingly. On purchases that are important to me, I often read multiple reviews across all “star” levels plus I read reviews outside of the purchasing platform. I know there’s bias and I try to diminish its effect by expanding my research. The challenge remains getting people to provide valuable reviews without bias. Social media has proven that people like hearing themselves talk so I’m not sure incentives are necessary but maybe a “reviewer rating system” might work. Sadly, I’ve never reviewed any of my purchases but promise to do so this weekend on at least 3 of my most prized purchases – it’s time to give back!
If only a YouTube reviewer has a following that they get paid for, are you incentivized to write happy reviews?
I have no doubt an incentivized review is biased. However, I believe there’s a difference between a review, biased as it may be, and the astroturfing that I see online. A review that accurately describes an item’s function and form can be very valuable, especially for big-ticket items. I shop for photography equipment online, and without incentivized reviews, especially early in a product’s lifecycle, I would have nothing to go on to make a purchase decision.
Identifying the reviewer as having been incentivized is helpful, but a poorly written review that hypes products without providing any real information, those reviews tend to be really transparent, and most shoppers have learned to spot them and ignore them.
Reviews are tricky these days. If a product has a solid five star rating I tend not to believe it. As a consumer I look for Amazon’s verified reviews because they connect the review to an actual purchase.
Online hotel reviews like the ones on TripAdvisor are invaluable when you spend a lot of time on the road. It’s easy to make a place look good with camera angles, but the actual traveler photos don’t lie.
People reading reviews expect them to be unbiased and independent thoughts from the reviewer. Any perceived influence created by an incentive from a product seller will cause suspicion by the buyer looking at a product review.
Making reviewers reveal incentives is good for honesty, but it might not stop all fake positive reviews. Even if they say they got free stuff, some reviewers might still exaggerate.
Stricter enforcement, along with diversified review sources and algorithms to detect suspicious patterns, could complement disclosure measures for more effective review authenticity.
Retailers can encourage reviews through ethical and effective means that promote genuine feedback including product quality, user-friendly platform, post-purchase emails, transparency, incentive, response engagement, and other innovative methods.
Well the study answers the question, doesn’t it: “reduce”, yes…”eliminate”, no.
And more importantly, I think it points out bias is inherent in ANY retailer directed evaluation process. That having been said, I think we’re perhaps focusing a little too much attention on metrics like how many stars are given and ignoring the role reviews play in simply increasing the amount of information available: as has always been the case, people are free to (selectively) make use of what is offered…or ignore it altogether.
Authenticity is the key and foundational element of value in ratings & reviews, and user-generated content more broadly. Without confidence that UGC reflects an authentic purchases authentic perspective about a product, the whole value of such content is lost.
That said, simply incenting a consumer to share their perspective by providing the product for free does not inherently damage that value, They are free to give the product a good review or a poor one, and there is no incentive to give anything other than their true thoughts and perspectives.
Even though there may not be formal incentive programs, consumers still receive direct incentives shipped with products.
We have become desensitized to the mechanical disclosures of paid, partner or free goods in exchange for an “unbiased” review.
Reviewers know what’s in it for them if they give a positive review. Consumers know there is some bias. Exploring motivations for prividing honest, uncentivized reviews is a worthy undertaking by retailers and merchants.
Yes, they’re still biased. Whether they add value to the customer is in the details. Are they high level and generic, or are they specific? People want to read details about how a product is being used, frequency of use, wash and wear details, size/fit based on specific body type, etc. Using promotions to drive additional details in reviews would add far more value to customers vs just driving volume of reviews.