Are humans biased against AI-driven recommendations?

Photo: @g_dezigner via Twenty20
Dec 09, 2020

Although research has shown bias can creep into algorithms, humans are also found to be biased when it comes to artificial intelligence (AI). The negativity some humans feel may even be limiting the potential of AI-driven recommendations, according to a university study.

Relying on data from over 3,000 study participants, researchers from Boston University and University of Virginia explored the “word-of-machine effect” or what the authors assert is the false belief that AI systems are more competent than humans in dispensing advice when functional and practical qualities (utilitarian) are desired and less competent when the desired qualities are experiential and sensory-based (hedonic).

“The fact of the matter is humans are not necessarily less competent than AI at assessing and evaluating utilitarian attributes,” the authors wrote in the Harvard Business Review. “Vice versa, AI is not necessarily less competent than humans at assessing and evaluating hedonic attributes. Indeed, AI selects flower arrangements for 1-800-Flowers and creates new flavors for food companies such as McCormick.”

Among their findings:

  • Utilitarian vs. hedonic: From a marketing perspective, due to common consumer biases, playing up AI over human assistance tends to be more effective for selling products in which utilitarian/functional features are more important. Emphasizing human over AI assistance was more beneficial for products in which hedonic/experiential features are more important.
  • Unique vs. common: One study indicated that when consumers wanted recommendations matched to their unique preferences, human recommenders  were preferred over AI regardless of hedonic or utilitarian preferences. The results suggest customers known to be satisfied with “one size fits all” recommendations (i.e., not in need of a high level of customization) would be open to relying on AI-systems.
  • AI-human hybrid: Companies selling products that promise more sensorial experiences (e.g., fragrances, food, wine) were found to embrace AI’s recommendations as long as AI works in partnership with humans. Researchers wrote in a press release, “When AI plays an assistive role, ‘augmenting’ human intelligence rather than replacing it, the AI-human hybrid recommender performs as well as a human-only assistant.”
  • Removing bias: Pointing out the “incorrect lay belief” about the competence of AI versus humans in providing advice resulted in more favorable perceptions of AI recommendations.

DISCUSSION QUESTIONS: Does it make sense that consumers would be biased about AI-based recommendations? Should marketers be playing up AI-based recommendations for some products and hiding them in others or seek to reduce biases?

Please practice The RetailWire Golden Rule when submitting your comments.
"I think AI has its uses but the perception of what it can actually do is WAY overblown. "
"The better question might be “are AI-driven recommendations biased against humans?”"
"AI shouldn’t be thought of as “Artificial Intelligence,” but more as “Augmented Intelligence.”"

Join the Discussion!

14 Comments on "Are humans biased against AI-driven recommendations?"

Sort by:   newest | oldest | most voted
Paula Rosenblum

I think AI has its uses but the perception of what it can actually do is WAY overblown.

And now we have a problem. Consumers believe it can do more than it can at the same time as they see the evidence that it can’t. For example: the Facebook algorithm flags a photo of a puppy as “inappropriate” and the users just laugh. And yes, this really happens.

So given all of that, will consumers be biased? Yeah. And no, marketers most definitely should NOT be playing it up. Maybe some version of Machine Learning, which is the same but subtly different.

Matthew Pavich

As with numerous technologies, it makes sense that some people are more open to adopting AI than others. The reality is that it is here, it works and people use it all the time and accept that algorithms are used (i.e. Uber, flights, hotel pricing, etc.) when making purchases. I have absolutely seen AI transform retailers in the pricing space and make their pricing and promotions more effective. I have also seen its benefits to the consumer by identifying opportunities to price more aggressively on essential items that help shoppers stretch their limited budgets. Ultimately most retailers will benefit from incorporating AI but will need to be savvy in their messaging and may choose terms like “consumer informed or consumer derived” instead of “AI or machine-learning.” Listening to what your consumers want has always been essential to retail — AI is simply a more effective, more dynamic way to do so!

Suresh Chaganti

It is important to remember that models are trained and the results are interpreted by humans. Bias can creep in. A spam filter is something we use every day. There are false positives and false negatives all the time. The machine will learn over time, but the human is responsible for initial modeling, setting training parameters to instruct what is spam and what isn’t, and retraining when new types of email patterns emerge.

Most marketing falls into a similar category. This is the reason why unbiased data science and objectivity is so important as the fundamental trait. Involuntary misses, lack of rigor in modeling, training, and testing also introduce errors and reduce accuracy. Consumers could attribute that lack of accuracy to bias.

Di Di Chan

Humans input databases that train AI. The quality of the AI recommendations will depend on the quality of their input. Likewise, AI will also learn whatever biases they are taught to mimic. Not all AI systems will perform the same because they are not getting the same input.

Amazon generates 35 percent in sales through its recommendations engine. To compete with that, you either have to have a big database to train or smarter inputs. For example, Halla’s specializes in AI recommendation for the grocery space. Their AI recommendations engine is unique because they use natural language and recipe input to generate much better recommendations results than average. Marketers should test different approaches and pick based on performance and results.

Bindu Gupta

Consumers could be biased about AI-based recommendations due to any poor or irrelevant experiences they had with their results such as irrelevant product recommendations churned out by recommendation engines. It is important to remember that the effectiveness of this technology would depend upon the way it is programmed. Marketers should highlight AI-based recommendations if they are confident that consumers will get relevant results and have a positive experience.

Katie Hotze
1 year 11 months ago

Busy people don’t want to think – they’re too busy! This is where AI excels in delivering thoughtful recommendations. Conversely, there will always be a deeper need for personalization and input from a human component.

Ricardo Belmar

This is really about emotional attachment to products and brands recommended to consumers. If a consumer needs to feel an emotional attachment to something they are about to buy, they want to know a human was involved in any recommendation process over AI. However if we’re talking about purchasing something commoditized and not emotional, say motor oil for your car vs. a pair of shoes, then sure, consumers will feel better knowing an AI that purportedly analyzed data to derive a “best option” to buy. Of course one shopper’s emotional attachment is another’s commoditized/utilitarian buy. So what does this mean for retailers? Like most marketing strategies, a balance must be struck and it might not be in your best interest to highlight the involvement of an AI to the maximum extent. This is where humans have to make a judgment call on how to proceed, regardless of what the AI is telling you!

Brandon Rael

Purpose-driven and value-added AI recommendations are always welcome, especially if they add to the overall customer experience. Whether we realize it or not, we have an internalized subconscious bias about anything in life. The AI-powered move to drive enhanced personalization may veer too far and, in the process, alienate loyal customers.

Retail has always been about the blending of art and science. Automation and AI certainly have their place and will learn and improve over time as the algorithms mature. However there are the emotional and nostalgic attachments that AI may not always consider in the recommendation engines. There are still personal biases and intuition that play a significant role in the purchasing process.

Cynthia Holcomb

The better question might be “are AI-driven recommendations biased against humans?” Given a single programmer can unilaterally decide what is the correct answer to what an algorithm is solving for, using “selected” data inputs to train and educate the algorithm[s], inherently breeds an algorithmic bias executed as an Ai enabled recommendation [decision].

Ananda Chakravarty

Sometimes we forget the real value of AI for retail is scalability. AI allows addressing broader sets of customers and engaging at a far deeper level than human individuals. It would take an army of humans to build personalized recommendations. AI does this faster and on the fly with dynamic changes included. There can be quality lapses and mistakes, but it doesn’t make up for the fact that the human would be able to create a custom flower basket for 20 in an hour while AI can do this across 20,000.

Humans can certainly do a better job, but AI can do an adequate job for thousands more. So sure, there will be bias against AI. But there will always be a niche market for those who get excited about being served by a machine.

James Tenser
The focus of this particular study from Journal of Marketing specifically addresses how shoppers feel about and trust AI recommendation engines. While an important use case, this represents only a fraction of what AI can do. AIs are not automated tools for executing human-created algorithms. They absorb data and model it using purely mathematical methods. When information about an individual is compared against the model, certain probabilistic predictions and recommendations are possible — including “people like you also like product X.” The study design used an experimental method to test how well shoppers trust recommendations they are told were AI-generated. It discovered measurable preference for human-moderated recommendations over pure, automated ones. In the real world, however, online recommendation engines are seldom, if ever, moderated by humans. Consumers either learn to trust the results they see after repeated exposure, or they learn to be skeptical about them. Here’s what I think is important about these learnings: Personalization-at-scale requires a high degree of automation which AI can provide. If consumers have already learned to mistrust AI recommendations… Read more »
Ralph Jacobson

All of us have inherent biases. Here’s a great book on that. AI shouldn’t be thought of as “Artificial Intelligence,” but more as “Augmented Intelligence.” It is simply helping humans make better decisions faster. I wouldn’t highlight the technology in the retailer’s recommendations. I would compliment the shopping journey with helpful, relevant suggestions that can be very targeted individually with AI technologies available today.

Casey Craig

Consumer bias towards AI-based recommendations and personalization is a real and growing phenomenon. The biases (real or perceived) are appearing in personalization and recommendation engines within e-commerce sites, as well as loyalty programs. Quality AI output is a key component of maximizing outcomes (brand loyalty, revenue, and profits).

Marketers should strive to reduce bias within algorithms as well as the data used to train the algorithms, thereby minimizing the potential of bias seeping into one or more segments of a company’s customer database. This can be accomplished by ensuring the quality of each customer segment’s demographics and attributes are correctly represented.

Rodger Buyvoets

It’s interesting that people can apparently distinguish between “one size fits all” and “unique” recommendations, of which the latter tends often to be absent. This is very common for recommendations; they tend to “grow” towards a self-fulfilling prophecy, i.e. only recommending items that are often bought. This can be solved by either adding perturbation in the data sets and/or adding context (psychographic/ social behavior), thereby making the recommendations more “unique.” Working with AI for a while now, I’ve noticed how retailers want to have some control on recommendations for filtering/adjustment. This bias is called “boost & bury”; some items — though strongly favorish and popular — are hidden (buried), and others — more rarely items — are boosted to enforce uniqueness.

"I think AI has its uses but the perception of what it can actually do is WAY overblown. "
"The better question might be “are AI-driven recommendations biased against humans?”"
"AI shouldn’t be thought of as “Artificial Intelligence,” but more as “Augmented Intelligence.”"

Take Our Instant Poll

Do you agree with the common sentiment that AI systems are more competent than humans in dispensing advice when utilitarian features are desired and less so when hedonic qualities are needed?

View Results

Loading ... Loading ...