Bias can be an unintended consequence of applying AI algorithms as the examples in the article show, and so practitioners must remain vigilant in reviewing and monitoring outcomes generated by these systems. AI and machine learning are still in their infancy and so to is retailers’ ability to uncover and fix biases. Like Amazon and others who are actively using and applying AI, this is still a work in progress.

The myth around AI is that a neutral “machine” independently develops its own intelligence. In reality, AI and machine learning, even instances that are to a strong degree “self-educating,” originate from human-derived algorithms. Humans are flawed and often from their early intervention into creating/training artificially intelligent systems, can introduce bias. I listed it as a probable issue with Amazon Scout in my article “Amazon is Scouting the Wrong Approach to Improving Product Discovery.”

I doubt that many AI systems anywhere are evaluating for much more than obvious bias because they are designed to enhance biased outcomes: meaning, designed to create outcomes to support a business case or need, which is a purposeful interest.

Retailers do need to use AI cautiously for both bias issues that can affect and offend customers and can affect profitability via missed opportunities and wrong assumptions.

Almost every technology and technology-based decisions have some unintended consequence. Since AI software does nor develop itself but, rather, it is developed by people, there is always the probability that AI will contain some human bias, whether intended or not. Of course, these unintended errors will be discovered — eventually, and corrected — hopefully.

As machines do not, as of yet, have volitional consciousness they are not biased by themselves. Bias comes from human programming or from machines observing human patterns of behavior which exhibit bias.

Sometimes this bias is useful. If I, as a consumer, have a bias towards buying certain products then that helps retailers target me more effectively. Sometimes, the bias is less helpful such as in recruiting or profiling people when there is no need.

They key thing is to ensure that algorithms are auditable and to review outcomes and to ensure that these comply to ethical frameworks and policies. Given that most retailers haven’t really fully got to grips with AI itself, I think exploring the ethical dimension is far from advanced.

Everyone should be highly concerned about latent biases. We don’t fully understand our own biases, and without that understanding, we can never assume that AI won’t “learn the wrong things.” This is why I think there will be an “AI 2.0” in our future – one that has been developed specifically to make its assumptions or lessons learned clear to an end user, who can then better monitor the things that AI learns. The reality is, we WANT AI to learn “ideal” behaviors, but we’ve released it into a world far less than ideal. And it will do with that data what AI does — draw conclusions and apply them going forward.

Garbage in — garbage out. If the data the algorithm is being trained on is not somehow scrutinized or scrubbed, it will likely be based on bias and bad assumptions.

The view of the world an AI has, is not the world; it’s its training data. The inscrutable nature of AI makes it difficult, or impossible, to understand prediction logic and to pinpoint flaws of bias; this is still very much a research problem. However, this is where I believe that diversity of thought among applied data scientist, data engineers, and ethics validators will pay dividends.

Artificial Intelligence (AI) is not really “intelligence” at all, but rather simply information, hidden in vast skeins of data, about what IS, that is not readily NOTICED by our limited minds. However, it is doubtful that that data accounts for all the biases produced by programmers from demonstrated human monocultures, politically, socially, etc.

Some of the biases may be historically justifiable, but maybe not, too. It’s a mess. But THAT’s the human race, that is on a largely unrecognized positive trajectory. See: Steven Pinker, “Is the World Getting Better or Worse?“

I agree with Neil’s comments. AI, at the moment, is another magic bullet that is perceived to allow us to alter retail. It’s not clear that this is actually widely in use.

It will be an interesting problem to solve once it has been deployed and is mature enough to truly support decision-making at an individual level, but meanwhile I’m happy to see it getting closer to an accurate weather forecast.

EVERYTHING not of nature, that is, built or manifested by human kind, came from imagination and thought. Those of us with a creationist bent believe even our natural world came to us that way. Algorithms themselves came into existence through the same magical influences that created the paintings of the great masters. Their “biases,” the way they see and experience the world, is what makes those works treasured for all time. So what is the difference between the creation of AI technology and these masterpieces? The answer lies in the “Why?” da Vinci, Rembrandt and others, I believe, were channels of what they produced. To get more into the science side, this was true of Fleming and Tesla too. They did their work because their destiny called them to do it. And that is what we’re rapidly losing today. Everything that will ever be possible already exists on a non-material plane. Our work, as creative instruments of the future, is to pick up on which possibilities have our name on them. When we find them,… Read more »

Let me start by questioning whether or not anyone — other than zealots — believes AI “promises to eliminate the inefficiencies that come from subjective human assessments and assumptions.” AI recognizes patterns in data. That’s like thought, but it isn’t totally objective thought. So if there are biases in the data, they will be mirrored in the analysis. These are not independent systems, and they are only as objective as the developers who set them up and the data fields they are analyzing. On any given application they aren’t inherently any more or less biased than a human being, just faster and less prone to computational error.

Retailer hiring algorithms may be where the most egregious AI errors lie. The excellent book “Weapons of Math Destruction” by Cathy O’Neil discusses hiring algorithms for their secrecy and the complete inability to improve the algorithms by testing.

For proof, we have the recent choice by Amazon to stop using their AI hiring algorithm because it showed bias against women.

But I don’t think the problem is fixable. These biases are hidden and nearly impossible to test. AI can ONLY be led to succeed when the algorithms are trainable by evaluating results. But in hiring software there’s no way to train with “person X was hired elsewhere and performed outstanding work.”

AI is not biased, it is the data that is biased and the humans that created the data. If human decisions have been biased in decisions that impact data, then the output from AI will perpetuate that bias. Garbage in, garbage out. That old phrase is as appropriate today as it ever was. Systemic bias needs to be scrubbed out of the data so neural networks will apply the proper framework of algorithms to make AI unbiased.

Retailers need to be aware of the potential mistakes that AI can make based on historical data and biases and monitor the results of AI to help catch faulty logic. Humans are not perfect and neither is AI.

As already noted, the bias is likely in the data, which influences the algorithms. Next logical question, if AI “judgement” is based on historical patterns, where does innovation come from? In merchandising, the challenge has always been the self fulfilling prophesy. Don’t retire the creative types in your organization yet …

What a great set of thoughts already. Big concern — yes. Biases, if uncovered, can topple brand value rapidly, especially as it dehumanizes engagement with customers. We are still at the early stages of AI and the problem that occurs is that our bias corrections are not objective enough to be bias free, so we just introduce another set of biases into the training data or apply fixed business rules on top of results, diminishing the value of AI tools to uncover things that humans wouldn’t normally find easily. AI is quite promising, but we’re not at a point where the concerns are well founded nor can we deliver results that can be used readily in retail — except in select cases such as logistics or demand planning.

Like any new science, AI is going through learning lessons, including latent biases. Researchers are working on many techniques to eliminate the bias. By studying the results of AI, biases will typically be found if they exist. Retailers are probably not performing much analysis to determine if biases exist as any AI bias will have little impact on people. The biases may cause the retailer to have the wrong assortment or products in store at the wrong time, nothing that will effect the shopper personally.

There should be a big concern about AI being biased or making bad decisions based on biased programing or human behaviors that cause bias. It has been my position for several years that AI is not ready for advanced decision making. It can help, but it can’t be used (yet) for big issues and decisions. It’s perfect for lower level support and functions. In the HR world, it can be used to create a streamlined application process, help with repetitive tasks and more basic functions. To use it as the sole decision maker for hiring is too soon, if ever at all.

AI is still relative new and still being developed as a solution. Inherent biases are hard to unearth with people which means AI-solutions are bound to have bias (even if vetted). The key is to find and understand how the biases impact the end result.