Big Data is done, put a fork in it
Rick Ferguson, CLMP
Through a special arrangement, presented here for discussion is a summary of a current article from The Wise Marketer, a website and newsletter serving the global loyalty industry.
About a decade ago, the phrase “data is the new oil” swept the globe as the twin corporate power centers of IT and marketing realized their companies had more data than they knew what to do with. “Drinking from the fire hose” became a metaphor for the struggles of extracting actionable insights from data.
About five years ago, the usual suspects in IT consulting and cloud-based analytics began to trumpet the phrase “Big Data” to sell into companies hoping to extract that resource.
Now, as Slate’s Will Oremus points out, the phrase Big Data has become passé — in part because we just call it data now, and in part because the rush to rely solely on data for business decision-making has often revealed the limitations of data-based decisions.
Our over-confidence in these tools often hinders our ability to see the forest for the trees. We often fall victim to what data scientist Shane Brennan calls the “Ten Fallacies of Data Science.” For a variety of reasons, the data may be inaccessible, indecipherable, outdated, lacking enough granularity for analysis or prohibitively expensive to get or have processed, Mr. Brennan contends. Often, a marketer’s lack of understanding or pre-conceived notions leads to poor decisions.
“Garbage in/garbage out” results in marketing campaigns or loyalty offers that are ineffective or even harmful to your business.
Should all data analysis be eschewed in favor of, say, gut feel, astrological charts or throwing darts at a board? Not at all. Data analytics is being successfully used to fuel personalization, construct relevant offers and build differentiated experiences.
But by understanding the limits of data science, your efforts on data can best correlate with current customer value or be most predictive of future customer value. Call it “small data” or “customer-centric-data,” or just continue to call it data. If customer behavior shifts in a profitable direction because of your analysis, then you’ll know you’re on the right track. Be diligent, question your assumptions and be aware of your biases. Big Data may be over, but data science, like any scientific pursuit, is forever.
- The rise and fall of Big Data – The Wise Marketer
- How “Big Data” Went Bust – Slate
- The Ten Fallacies of Data Science – Medium
DISCUSSION QUESTIONS: Should “Big Data” be retired as a progressive movement in marketing? Do retailers and brands generally understand the limitations of data science within their organizations? What advice would you have for marketers attempting to best leverage the data at their disposal?
Lyle Bunn (Ph.D. Hon)Strategy Architect – Digital Place-based Media
Join the Discussion!
36 Comments on "Big Data is done, put a fork in it"
You must be logged in to post a comment.
You must be logged in to post a comment.
President/CEO, The Retail Doctor
This is semantics. We use Big Data now to refer to AI, machine learning and predictive analytics in one easy term. Watson is much more than just “data.”
Principal, Your Retail Authority, LLC
Yes, Bob, it is semantics. Data in retail will always be fuel behind almost everything we do. It’s how smart we get in using it. For my 2 cents.
Co-founder, RSR Research
I don’t think “Big Data” was ever well defined, and the systems that might actually make use of the insights were not in place in retail operations.
And then there was omnichannel, which overtook corporate IT spending (along with security).
I think the biggest issue with retailers and brands is not that they don’t understand the limitation of data science — it’s more that the data scientists have gone into financial services, where they can make real money. It’s on the vendors to provide the science and make it easily digestible for the retailers, et al.
Principal, Retailing In Focus LLC
If you read yesterday’s Wall Street Journal interview with the chairman of Fast Retailing (Uniqlo), you might have seen this perceptive comment: “Data would never substitute the merchant. How do you interpret the data? That’s the merchant’s skill set. You need to uncover the insight that is buried in the data and the merchants need to uncover it. Even if you employ artificial intelligence to help you, the numbers [don’t tell] the future.”
His point is well taken: No matter how much importance a retail organization places on its ability to extract data from its transactions, the information means nothing if it can’t be turned into action — and some of that decision-making rests on instinct and experience.
That being said, declaring that “Big Data is dead” is an overreaction. The phrase itself may be overused, but data science is alive and well in the interest of smarter merchandising decisions, loyalty programs and so forth. Would Amazon be where it is today without groundbreaking use of data to develop its predictive technology? I don’t think so.
Principal, Retailing In Focus LLC
And just to add to my comment: I teach an undergrad retailing class at UW-Milwaukee and one of my guest speakers this term is a data scientist whose firm works with multiple omnichannel and e-commerce retailers. He made the point that 90 percent of the world’s data has been generated over the last two years, and the trend toward accelerating data generation continues.
Can anybody honestly say that “data science” is dead, even if “Big Data” is a cliche?
Founder | CEO, Female Brain Ai & Prefeye - Preference Science Technologies Inc.
Good points. I agree with Mr. Yani’s comments as well. What is missing? Data scientists are working to interpret data on a subject requiring deep knowledge only an experienced merchant has earned. Thus the disconnect. To cross this chasm will require specialized merchants to work hand in hand with data scientists.
Managing Partner, Advanced Simulations
Back in 2013 I delivered a paper at ESOMAR’s congress on why Big Data was not such a big deal — this echos a lot of what I said at the time. I’m not sure this was ever a “progressive” movement. We were sold a bill of goods on possibilities that turned out not to exist. I’ve always maintained that marketers and retailers need to focus on the questions they would like answered and let their researchers find the best way to answer them. We shouldn’t put the cart before the horse, trying to leverage the data we have, just to try and leverage it.
President, The Ian Percy Corporation
Excellent response, Stephen!
Co-founder, CART
Well said. Data and insights for the sake of data and insights doesn’t net any impact on the business. It’s what is done as a result that makes the difference. Having the business look at what questions to ask connected to actions they could take is definitely the best approach.
Vice President of Marketing, OrderDynamics
Thanks for retiring the term. Big Data is probably the most overused and abused term in the business world. I have heard VPs talk about the critical importance of it, and how every firm needs it, but they did not really understand much about the hows, whys, tactics or implications.
Data is messy. The important thing is that executives understand that you still have to make a decision on the data you have at hand, and it still will often come down to a judgment call. The worst thing you can do is to fall into an analysis-paralysis vortex. Don’t let this happen to you! There is a point where you have analyzed the data enough and it is time to make a judgment call. By the time you have full clean data, the conclusion is usually obvious. Your competitors have already long since taken action while you sit pondering what happened — in the dust.
Global Retail & CPG Sales Strategist, IBM
I’m just a bit concerned about stating that “Big Data” should be retired without the intentional, highlighted hand-off to another term, like, “GINORMOUS DATA.” Retailer and CPG brands haven’t even scratched the surface of capturing and deriving insights from the more than 80 percent of all data which is “dark” and invisible to current systems in these enterprises. Capturing this data is the first step in acknowledging what the current limitations of data science really are. From there, there’s a ton of work to do still. Let’s get going!
President, The Ian Percy Corporation
You trigger the thought that “information” and “insight” are totally different things though often confused.
Founder and CEO, Bobsled Marketing
I look to arguably the best example of a company who obtains, compiles, and uses data to its full advantage: Amazon. As a core leadership principle, Amazon uses data to take huge bets and lead decisions. And when the data doesn’t support an idea, the venture is quickly killed and moves on. It seems so much part of the DNA of the company that giving it a special title is pointless. It’s just the way Amazon does business.
Professor of Marketing, The Wharton School of the Univ. of Pennsylvania
I’m as strong an advocate as you can get for “Small Data,” i.e., squeezing as much value/insight as possible about simple transaction log data before turning to more elaborate “Big Data” schemes. But that doesn’t mean that Big Data is done — quite to the contrary, the real era of Big Data hasn’t even started yet. Once we master the art and science of Small Data, we’ll finally be in a position to start to properly leverage Big Data, in ways that most retailers can’t even imagine today.
It’s a lot like CRM: people were writing off CRM systems (literally and figuratively) 15 years ago because they weren’t yet in a position to make strategic decisions that required it. But today we know that CRM is the bare minimum pf what you need for customer-centric success. Big Data will have a similar resurgence in a few years.
But first retailers really need to get started with the Small Data revolution that is right at their doorstep and ready to happen …
Managing Partner, Deeb MacDonald & Associates, L.L.C.
The new term should be “Smart Data.” Companies that have learned how best to efficiently mine their data for better sales and profits, customer retention, better logistics operations etc. are already on to this “smart” usage. There are many examples — from dunnhumby to Watson — of businesses that understand how to utilize their information for results.
Strategy & Operations Delivery Leader
Similar to the apprehension retail industry experts have about the term “omnichannel,” Big Data now shares a similar distinction.
Big Data is a term that was intended to simplify the complexity of all of your data sources, consolidate it, and present it back in a form that is both consumable and able to be operationalized.
Essentially we should retire this term, as it overflows into the predictive analytics, augmented intelligence, cognitive data and artificial intelligence arenas, which are far better defined and are undergoing significant evolutions.
Strategy Architect – Digital Place-based Media
A stake to the heart of the term Big Data is not going to kill the essence of managing by information. Let it morph into a descriptor of its business value rather than what it is, which has driven investment and has been the enterprise call to action, through which abilities and incapabilities are being continuously revealed.
Retail Solutions Executive, Teradata
Big Data as a term might be passe’ but the use of Big Data is only going to grow with the adoption of machine learning tools and AI (coupled with prebuilt retail data models) to harvest greater volumes of insights from growing volumes of data that are bigger than what data scientists could ever dig through for non-critical or “cold” data needs. THAT is where the term Big Data makes sense. The key is to have a clear data strategy and architecture that allows for smaller and more critical data sets to be analyzed by the data science teams within retailers for better results.
President, founder and CEO Interactive Edge
President, The Ian Percy Corporation
I’m with Stephen Needel on this. Retail (and most other organizations) are prone to a “band-wagon” fetish. Too often we jump into things like Big Data out of desperation with little insight or foresight. Too much fertilizer is called weed killer. Too much data is called stagnation and paralysis.
Founder | CEO, Female Brain Ai & Prefeye - Preference Science Technologies Inc.
The best way for a retailer to leverage data? Think outside the box of AI, ML, segmentation, visually alike, collaborative filtering and the like. The best data to leverage? Customer preference intelligence. Daily individual customers tell a retailer what they like and do not like. Customer preference microdata, cognitively translated by an expert system able to understand the inter-relationship between thousands of sensory attribute data points in a product to uncover the specific, individual sensory preferences of a human customer; in other words, like a human thinks and processes data.
AI and machine learning are still in their infancy when it comes to processing data like the human brain, lacking the ability to understand the subjective nuances between a person and a product to truly leverage data in personalized recommendations. Retailers have reams of micro customer preference intelligence accessible right before their eyes.
Director, Solutions Marketing with Alteryx
Marketers looking to best leverage the data at their disposal should collaborate across agency, internal IT and data science around their customer journeys or customer experiences. The label “Big Data” doesn’t really matter — it’s all about the analytics maturity of a retailer and how they are organized.
Too many retailers have bolted on data science, marketers don’t speak their language, and the data that agencies manage remains fenced off from transactional and customer interaction data created by internal systems. Speed, speed, speed and test, test, test, while quickly deploying to production or scaling the highest value use cases has got to be a priority.
Advancements in AI are like an octane booster to get retailers moving faster and taking advantage of methods they haven’t yet productionized like relatively straightforward personalization. The leaders are extending the gap here and most retailers have got to catch up quickly.