How much Big Data do retailers really need?
Brian Kilcourse
Through a special arrangement, what follows is a summary of an article from Retail Paradox, RSR Research’s weekly analysis on emerging issues facing retailers, presented here for discussion.
A recent RSR survey found half of retailers had implemented capabilities related to capturing and analyzing “Big Data while less than a quarter claimed to be “satisfied” with what they had implemented.
So are retailers just in the very early days when it comes to truly leveraging non-transactional data or just too slow to change?
Predictive analytics represent a huge leap forward because it takes into consideration not only internal transactional data but path-to-purchase and social data as well as exogenous data like weather, competitive, consumer psychographic profile, calendared events (such as concerts or sporting events), and manufacturer and supply chain data.
Consumers want more relevance, and that also means not slogging through mountains of merchandise they don’t want to get to the items they do want. That’s what localization of the offer is all about.
But one-on-one marketing isn’t the right answer for everyone. Retailers need to answer these questions:
- What level of “intimacy” is needed to know what you need to know to be relevant? The granularity of customer-specific data needed to make a highly relevant offer is closely related to the specificity of the need (for example, offering basic commodities doesn’t require much customer specificity);
- What level of relevance is needed to be relevant? This relates to the granularity of the offer itself. Extreme localization isn’t necessary for every brand;
- How much data do you really need? Or how much analytical sophistication do you need to have to apply what you know to be relevant to customers?
In a way, the world of possibilities that Big Data represents might be distracting retailers from the real question, “What value are you trying to deliver?” Getting an answer to that has to come first, and you don’t need technology to answer it. But you may need technology to deliver it, and do need technology to tell you whether or not you’re getting the job done in a way that will satisfy customers and generate profits. That’s when your version of Big Data becomes important.
Discussion Questions
DISCUSSION QUESTIONS: How can retailers get their heads around Big Data? How can they determine what level of granularity they need to offer the needed relevance to consumers? Will Big Data force retailers to change their business models?
Poll
BrainTrust
Anne Howe
Principal, Anne Howe Associates
Adam Silverman
SVP Marketing, Theatro
Use of corporate data assets is a regular topic among the retailers I have dinner with. Most say that there are multiple, disparate analytics platforms across their companies and no one is in charge of unifying those data points into actionable information. One company said that they couldn’t identify their customers, despite the fact that 100 percent of transactions are delivered to customers’ homes. The industry has a long way to go.
This question varies depending on the maturity of the organization. Larger companies tend to be well down the path with the right skills and technologies, but often struggle because not enough business sponsors participate in the process. Smaller companies struggle to move beyond very simple analysis because the skills required to drive the analytics are scarce. In either case, my view is that the rapidly maturing market for more packaged approaches to Big Data analytics helps organizations involve critical business stakeholders at scale.
In most cases, scaling value creation around Big Data is the hardest part and is why the market is sort of teetering. In practice, this means new Artificial Intelligence capabilities that are embedded in the processes and applications used by business people every day. It’s here that the value renders at scale and you create the type of people who appreciate and have interest in engaging in more custom analytical projects.
Sometimes the problem is the answer. If the problem is that retailers aren’t finding value and joy in Big Data, maybe the answer is to stop spending all your time, energy and money on trying to do so.
Look … weedkiller is, fundamentally, too much fertilizer. Too much fertilizer kills plants. Too much data just may kill retail.
At some point we need to ask: Is Big Data making retail a more energizing, fun and rewarding way to spend one’s life? Or is it exhausting us all?
Retailers looking to get more value from Big Data may want to re-consider the effort by putting the shoppers and their pain points at the center of a Big Data project. Resolving pain points usually gets shoppers to what they want and need sooner, leading to more sales and shoppers who are happy to be return customers. It’s not simple, but if customers help define the value they’re looking for, it’s not rocket science for retailers to figure out how to deliver it.
I’m with Anne. Understanding big data is only half of the equation; effectively using it to improve the customer experience is the real game. Determining what level of data collection is necessary for it to be actually actionable is a good place to start. Incorporating more detailed data, reporting and actions can come down the road. Several AI tools out there can shortcut much of the heavy lifting to get to more relevancy, faster.
Data needs and opportunities change over time with the business, and the great frustration is not in what is collected but in what has not been collected that can reveal historical trends. A top-line number such as traffic raises the questions “who are they?” and “when do they visit?” which then raises the question “what do they do when they visit?” Therein is the granularity of data that informs stocking, staffing, pricing and promotions. The same is true of conversion, the other important number that drives situation analysis.
Your comment reminds me of a book I wrote a while back, Lyle. I was going to title it “The Most Difficult Question in the Universe.” I didn’t, but I did ask a bunch of elementary school kids what they thought the most difficult question was. The best of them was from an eight-year-old girl who suggested: “Who are you and what do you want?” She’d make a great contribution to retail research today! 🙂
Big Data is a means to an end, and when companies treat it like an end they are likely to be disappointed. What I’ve seen in the number of initiatives I’ve been involved in is that focus on a specific outcome is a critical, clarifying element of any Big Data initiative.
In most cases a lot of data exists within retailers but it is not used in efficient ways to generate results. Data streams come from different parts of the organization. Unless there is a good system in place to aggregate the data, run automated processes and use AI and predictive models, it is very difficult to turn the data into actionable communication.
The most efficient use for Big Data for retailers is to communicate to customers using mass personalization tools that take into account purchase history, consumers behavior, demographics, churn and loyalty patterns and generates shopper profiles or rather shopper “DNA” to the level of the single shopper or household.
These capabilities are crucial for retailers to compete in the present and future marketplace, and retailers that adopt these tools and adapt their business models accordingly will be the winners.
Rather than referring to this as Big Data, which is often misunderstood, leveraging consumer insights is the more relevant term in today’s age of the customer. However, in order for this to be successfully rolled out, across retail organizations, Retail CXOs have to drive the culture around customer experience, empowered by consumer insights and embedded within the DNA of the entire retail organization.
From the customer’s perspective, the expectations are higher than ever for personalized, frictionless experiences, which essentially are fueled by Big Data and customer insights. Additionally, the customer only sees one brand, regardless of what channel (physical or digital) they are ultimately shopping with. It’s extremely important for retailers to stay completely connected to the customers, especially as digital has influenced an estimated 49 percent of U.S. retail sales.
Most critical for the retailers is to leverage the Big Data/consumer insights and enable them to be operationalized across all the disparate areas of the organization. Where retailers stand with this all depends on their organizational culture, digital maturity and ability to be agile and innovative.
Retailers were sold huge ideas about how data would transform business. It never had that power — it’s just smart to rely on data if you can get it for a reasonable investment. These survey results suggest executives are sorting this out.
I’ve been reading Deming lately and highly recommend reviewing his detailed writing on the use of data. While he is one of the original data scientists, he is keenly aware of the limitations of data to reveal important truth. (He’s even observed that the important things are often in what can’t be measured.)
But in particular, he suggests that top executives must be thoroughly versed in challenging data — to challenge the limitations of how it was collected, where it came from, errors in collection, errors in assuming too much from too little, detecting assumptions by their teams that aren’t warranted by the data and many more cautions.
I concur. The degree to which a company benefits from Big Data will not be based on the sophistication of its systems, but the sophistication of top executives who know how to make decisions after reviewing data.
Retailers who haven’t embraced Big Data, and likely are not terribly customer-centric, need to start small, test and learn. There is a wide spectrum between hyper-relevance, which is the future and mass retail, which is the past. Navigating that spectrum in the proper direction requires leadership and investment in technology that many retailers have not wanted to do.
Looking at the industry today, there is a clear divergence between retailers that understand the value of customer-centricity and correspondingly are investing in Big Data and those who don’t. Amazon has reset consumer expectations in terms of relevance and a better customer experience and those two priorities — customer-centricity and leveraging data in tandem — are what power success today. Not just for Amazon but for others that are continuing to be successful in the face of competition such as Best Buy.
There is an abundance of new and transformative technology to enable retailers in their journeys for customer relevance and it’s easy to make the argument that it’s not too late.
This is all about actionable insights. Truly. There’s no problem grabbing the mountains of data from transactions. However 80 percent of retailer data is “dark data” that is virtually invisible to current systems. And the data that is being captured is not being analyzed with tools that create simple, intuitive next best actions. The latest machine learning technologies are helping retailers make sense of all this as we speak. Check it out!
The problem comes from believing that the data has value and therefore that value must be extracted. The data itself has no value. The answers to questions the data may be able to provide, on the other hand, may have tremendous value. Retailers seem to have given the data a life of its own rather than treat it as a tool. Hire a great analyst who understands the retail business and then ask them questions.
Simply put, the magic of Big Data resides in its ability to help us achieve retail’s greatest promise: aligning the brand to customer expectations. Properly implemented data collection and analysis is the only way to reveal what customers expect of us, which in turn will help us define assortments, locations, pricing and personalization.
Anything less than comprehensive and robust interrogation of customer expectations results in taking shots in the dark.
Retailers need to collect everything they can even if they don’t have a plan to use it today. There are a number of companies, such as Uptake, who will help retailers and brands make more effective use of the data even if they can’t do it themselves. While you can always build machine learning and rules to examine past transactions, you can never go back and get the data if you didn’t collect it initially.
Like many other technologies this is just reflecting the immaturity in the industry in making the best use of a technology to solve a problem. Perhaps the real issue for retailers is that they cannot define what the problem is they are trying to solve with big data. Many retailers are simply solving the wrong problems and assuming big data is a cure all for everything.
Data is just that — data. By itself it will not provide the insight needed to make intelligent decisions. We’re seeing retailers start to hire data scientists and other specialists to handle this for a reason — the expertise required isn’t native to their legacy organizations. If you think about which brands leverage big data better, it’s always going to be younger brands who were not burdened by legacy organizational structures and silos. Use of big data will only get better for retailers going forward, but we are definitely in the early days.
Brian, I always enjoy your viewpoint and love this conversation! As a proponent of “Big Data” I often omit the adjective. Simply put, retailers I talk to struggle with Data.
I spoke with a CIO in the specialty segment a few months ago and she said, “Eric I don’t have Big Data,” but further discussion revealed that she had complex data, unstructured data, and distributed data.
The challenge was to simply collect and manage all the data in a comprehensive and effective way. Only then could she and her business team achieve the personalization that is so important to driving basket size and conversion.
Regardless of the descriptor, data is becoming the new center of gravity. Maybe not through mass, but through variety. Collecting data in its raw form at detailed line item level and storing in a future-proof platform allows the business model to adjust.
Yes, retailers should continually evaluate and adjust business models. We are in an environment of changing demographics, demanding millennials with mobile devices, and a hyper-competitive digital arena. Whether a retailer changes or not, the marketplace is changing around them.
Many of these signals and data points are available by simply listening to social sentiment. It becomes critical to any forward-thinking retailer to harvest insight from these data points in a secure and scalable way. While data may not be big to some retailers, it most certainly will get there, and en route will certainly be complex!
Some comments here hit the nail on the head, but most tend to overcomplicate it. The answer is simple: it’s all about clean, complete customer-level transaction log data. Retailers have it much easier/better than most other sectors, but they are pretty bad about using it effectively.
Job #1 is to get up to speed with granular transaction data before trying to master other data sources or seeking deeper insights that aren’t supported well by standard data structures.
Big data is a sexy term right now. Retailers want it. We’ll even pay big bucks to get it. The problem is when we do so before deciding what *exactly* we want to learn from it. Big data is only useful when it’s collected — and presented — in a way tailored to answer a specific question. The more you can restrict what you’re gathering in the first place, and then even further restrict what you make visible, the more useful insights you can gain from it. For example, your marketing automation software should be tracking every item every prospect looks at on your retail website. But if you displayed all that information it would be overwhelming and most retailers would give up looking at it pretty quickly. Too much noise to weed through! But if you gather that information, but only *display* it for consumers that are most likely to purchase (say those that have de-anonymized themselves) then you’ve got a useful dashboard of information to view and use.
Guess what? Big Data existed 100 years ago, back in the days of the general store. The proprietor knew everything about everybody — if they lost their job, if they were sick, if their sister was dating a bum. Or if someone was at the saloon bad-mouthing them. So they knew what products to recommend when — and how to price them. They knew when and how to say “You might need these bandages if you are buying these fishing hooks.”
We’ve now come full circle and Big Data can tell retailers exactly that. Yet few retailers fully understand or appreciate how they can use Big Data in 2017 to become the general store proprietor of 1917.
It’s a new world and they weren’t trained for it. But magically, the analytics that were deep in the retail memory of the general store owner — by virtue of deep personal and community insight, can be delivered by AI.
The analogy is not a stretch. In 1917 the proprietor had his own personal network. Everyone came into the store. Today, everyone enters the worlds of Facebook and other social platforms. So the data is there if retailers have the right tools to extract, analyze, organize and apply it.
Back in 1917, the general store owner also knew who mattered in town — who shaped opinions and whom others followed. Today we call them “influencers” and Big Data can tell us who they are, who they influence, what the trend drivers are and how to apply that in real time.
Retailers are struggling in the most competitive environment they have ever experienced. Loyalty is being challenged by easy access to competitive pricing. Inventory costs must be rationalized every day. And it takes Big Data to bring us back 100 years for the right answers.