This is semantics. We use Big Data now to refer to AI, machine learning and predictive analytics in one easy term. Watson is much more than just “data.”

I don’t think “Big Data” was ever well defined, and the systems that might actually make use of the insights were not in place in retail operations.

And then there was omnichannel, which overtook corporate IT spending (along with security).

I think the biggest issue with retailers and brands is not that they don’t understand the limitation of data science — it’s more that the data scientists have gone into financial services, where they can make real money. It’s on the vendors to provide the science and make it easily digestible for the retailers, et al.

If you read yesterday’s Wall Street Journal interview with the chairman of Fast Retailing (Uniqlo), you might have seen this perceptive comment: “Data would never substitute the merchant. How do you interpret the data? That’s the merchant’s skill set. You need to uncover the insight that is buried in the data and the merchants need to uncover it. Even if you employ artificial intelligence to help you, the numbers [don’t tell] the future.”

His point is well taken: No matter how much importance a retail organization places on its ability to extract data from its transactions, the information means nothing if it can’t be turned into action — and some of that decision-making rests on instinct and experience.

That being said, declaring that “Big Data is dead” is an overreaction. The phrase itself may be overused, but data science is alive and well in the interest of smarter merchandising decisions, loyalty programs and so forth. Would Amazon be where it is today without groundbreaking use of data to develop its predictive technology? I don’t think so.

Back in 2013 I delivered a paper at ESOMAR’s congress on why Big Data was not such a big deal — this echos a lot of what I said at the time. I’m not sure this was ever a “progressive” movement. We were sold a bill of goods on possibilities that turned out not to exist. I’ve always maintained that marketers and retailers need to focus on the questions they would like answered and let their researchers find the best way to answer them. We shouldn’t put the cart before the horse, trying to leverage the data we have, just to try and leverage it.

I’m just a bit concerned about stating that “Big Data” should be retired without the intentional, highlighted hand-off to another term, like, “GINORMOUS DATA.” Retailer and CPG brands haven’t even scratched the surface of capturing and deriving insights from the more than 80 percent of all data which is “dark” and invisible to current systems in these enterprises. Capturing this data is the first step in acknowledging what the current limitations of data science really are. From there, there’s a ton of work to do still. Let’s get going!

I look to arguably the best example of a company who obtains, compiles, and uses data to its full advantage: Amazon. As a core leadership principle, Amazon uses data to take huge bets and lead decisions. And when the data doesn’t support an idea, the venture is quickly killed and moves on. It seems so much part of the DNA of the company that giving it a special title is pointless. It’s just the way Amazon does business.

I’m as strong an advocate as you can get for “Small Data,” i.e., squeezing as much value/insight as possible about simple transaction log data before turning to more elaborate “Big Data” schemes. But that doesn’t mean that Big Data is done — quite to the contrary, the real era of Big Data hasn’t even started yet. Once we master the art and science of Small Data, we’ll finally be in a position to start to properly leverage Big Data, in ways that most retailers can’t even imagine today.

It’s a lot like CRM: people were writing off CRM systems (literally and figuratively) 15 years ago because they weren’t yet in a position to make strategic decisions that required it. But today we know that CRM is the bare minimum pf what you need for customer-centric success. Big Data will have a similar resurgence in a few years.

But first retailers really need to get started with the Small Data revolution that is right at their doorstep and ready to happen …

Similar to the apprehension retail industry experts have about the term “omnichannel,” Big Data now shares a similar distinction.

Big Data is a term that was intended to simplify the complexity of all of your data sources, consolidate it, and present it back in a form that is both consumable and able to be operationalized.

Essentially we should retire this term, as it overflows into the predictive analytics, augmented intelligence, cognitive data and artificial intelligence arenas, which are far better defined and are undergoing significant evolutions.

Big Data as a term might be passe’ but the use of Big Data is only going to grow with the adoption of machine learning tools and AI (coupled with prebuilt retail data models) to harvest greater volumes of insights from growing volumes of data that are bigger than what data scientists could ever dig through for non-critical or “cold” data needs. THAT is where the term Big Data makes sense. The key is to have a clear data strategy and architecture that allows for smaller and more critical data sets to be analyzed by the data science teams within retailers for better results.

I’m with Stephen Needel on this. Retail (and most other organizations) are prone to a “band-wagon” fetish. Too often we jump into things like Big Data out of desperation with little insight or foresight. Too much fertilizer is called weed killer. Too much data is called stagnation and paralysis.

The best way for a retailer to leverage data? Think outside the box of AI, ML, segmentation, visually alike, collaborative filtering and the like. The best data to leverage? Customer preference intelligence. Daily individual customers tell a retailer what they like and do not like. Customer preference microdata, cognitively translated by an expert system able to understand the inter-relationship between thousands of sensory attribute data points in a product to uncover the specific, individual sensory preferences of a human customer; in other words, like a human thinks and processes data.

AI and machine learning are still in their infancy when it comes to processing data like the human brain, lacking the ability to understand the subjective nuances between a person and a product to truly leverage data in personalized recommendations. Retailers have reams of micro customer preference intelligence accessible right before their eyes.

Marketers looking to best leverage the data at their disposal should collaborate across agency, internal IT and data science around their customer journeys or customer experiences. The label “Big Data” doesn’t really matter — it’s all about the analytics maturity of a retailer and how they are organized.

Too many retailers have bolted on data science, marketers don’t speak their language, and the data that agencies manage remains fenced off from transactional and customer interaction data created by internal systems. Speed, speed, speed and test, test, test, while quickly deploying to production or scaling the highest value use cases has got to be a priority.

Advancements in AI are like an octane booster to get retailers moving faster and taking advantage of methods they haven’t yet productionized like relatively straightforward personalization. The leaders are extending the gap here and most retailers have got to catch up quickly.

I can’t remember the last time I attended an industry event and have seen Big Data either in the keynote or in multiple breakout sessions. The term is used to cover an array of industry practices and just leads to more confusion.

I think the limitations of Big Data are now being accepted through dollars. Anything is possible if your budget is unlimited but to clean, harmonize and store data in a single repository costs a lot of money in not just upfront costs but ongoing maintenance.

The key to getting the most out of your data is to control what data goes into your database, create clear concise dashboards that aren’t in the weeds and keep out clunky, inconsistent data sources that just seem to break every other day.

The idea of “Big Data” stumbled because it was too much data for us humans to make sense of. That ushered in “Small Data;” more useable chunks of data that a data scientist could use to model with.

I think the recent advancements in machine learning now allow us to go back to making sense of “Big Data,” or lots of data. I’m not sure retailers are at the point of needing to understand the limitations of data science, as I think they still have not yet grasped the potential. Marketers should be clear on the business problem they are trying to solve for, then enlist data science to determine the right data to model in order to draw out insights.

Data does NOT think, nor is it creative. It is gas in the tank, not even the vehicle, much less the driver.

Thought provoking Rick, but I’m not sure most companies and brands, and retailers (other than AMZN and WMT especially) have fully embraced leveraging data to even get near a point where they have pushed the limits.

I’d challenge anyone reading this to name three retailers other than AMZN that demonstrate to you as a customer that they know “enough” about you to be truly relevant, improve your customer experience AND make you feel like that retailer is loyalty to YOU.

Most retailers still lack a single view of customers, much less one that is fully informed with the right data, including contextual.

All content will be data-driven. Regardless of the buzzword that’s used to describe or define the workflow, the fact remains that a broad spectrum of data needs to be leveraged to drive efficacy and optimization. It’s the only way digitally empowered shoppers will receive content that they value. It’s also the only way that brands can publish and deliver valued content.

History repeats itself, and tech is no exception. The rise and fall of the term “big data” is no different from the rise and fall of other terms “cloud,” “AI,” “Cognitive Computing,” “SOA,” “Micro-services,” et al. Each of these terms describe technical tool or concept with the potential to solve real business problems. Once people find the killer application of the tool or concept, the need to continue to use the term goes away.

Big data is far from dead — it’s just been renamed “AI” which is primarily using algorithms to process data. And that’s too bad. Because the term AI has led to an entirely new round of outsized hype.

Here’s what I believe we learned:

1. There is value to be learned by analyzing data.
2. It’s exceptionally rare that the value is large — as we know from decades analyzing data in direct response marketing it reveals small tweaks that are helpful but not earth shattering.
3. Big data and AI both require massive investments. And both return smaller incremental value. So we need to be very cautious about those investments.
4. Data/AI is not an end of its own. It merely complements other market intelligence like qualitative and quantitative research.
5. It’s deadly to retail when management lets the data make decisions.

The big lesson from the Data/AI adventure is that effective retail success when people make smart, responsible decisions after reviewing the available research, data, analytics, and human behaviors.

What we choose to call it isn’t what’s important — it’s all in the execution. Being able to apply big data based insights into tangible marketing and operational processes is still elusive for many if not most retailers. Most retailers I talk to will tell you they have more data than they know what to do with — it’s not about collecting and analyzing, it’s about applying the insight to their business. They just don’t have enough people in the right roles with the right experience to execute well. And that goes for marketing, operations, everywhere. There are small gains being made, but largely I believe there is much room for improvement. More data scientists would help, but as others here have pointed out, there’s more money in financial services for those folks than there is (so far) in retail.