Study: Big Data Remains Work in Progress
According to a survey, 60 percent of retailers are initiating strategies aimed at harnessing the power of ‘Big Data,’ but little progress has been made and the understanding of its impact remains low.
The "State of the Industry Research Series: Big Data in Retail" from Edgell Knowledge Network (EKN) was based on research using EKN’s Pulse360 methodology, including a survey of 75+ retailers, in-person interviews with senior retail leaders.
Highlights of the report include:
- Though 80 percent of respondents were aware of Big Data, only 47 percent are clear about its implications for their business;
- Nearly 70 percent of retailers surveyed already have a Big Data strategy or are building one. However, only 30 percent of retailers have done a Big Data proof of concept (POC), indicating that retailers are just beginning to understand the implications of Big Data;
- Asked their biggest challenge in managing data, 46 percent indicated handling data volume, 34 percent said handling data variety, and 20 percent indicated handing data velocity;
- Surveyed retailers identified marketing, merchandising and multi-channel as the three highest impact areas to consider for a Big Data pilot or POC. Not surprisingly, they also ranked these areas as experiencing the highest data growth;
- Business priorities, budgets and ROI were cited as the biggest inhibitors to investment. Thirty-one percent of surveyed retailers responded that ‘yet another’ analytics project will need a solid benefits argument.
- New analytics tools or software, and augmenting and training internal analytics resources were the most important factors towards building big data capabilities.
Based on the research findings, anecdotal experience captured from retailer and vendor interviews, in addition to guidance from EKN’s Advisory Council, the following approach towards Big Data was recommended:
- Take time to understand this space. Start by assessing Big Data maturity and do proof of concepts before you invest big.
- Identify areas of high impact opportunity, and build detailed use cases. Initially focus on three areas: pricing, segmentation and marketing effectiveness.
- Capability building and analytics training are critical to Big Data success. Identify where gaps exist in current capability; build specific recruitment and training plans.
- Create a comprehensive data strategy covering 3 core areas: customer data/master data management, data policy and process guidelines, and data pool use and sharing
- Focus on, and plan for, organizational change associated with Big Data and analytics adoption.
Gaurav Pant, Research Director, EKN, said the findings show that retailers are "beginning measured experiments" around that could spur growth in this area over the next 12-24 months. He added, "If retailers can avoid getting caught up in the semantics of Big Data definitions and focus instead on what decisions are valuable to their business, Big Data can be the game-changer it is touted to be."
- 60 percent of Retailers Are Initiating Strategies for Big Data, According to New Research Report From Edgell Knowledge Network – Edgell Knowledge Network
- State of the Industry Research Series: Big Data in Retail – Edgell Knowledge Network
- Future of Retail: How Companies Can Employ Big Data to Create a Better Shopping Experience – Time Magazine
- BrainTrust Query: Does Big Data Help Retailers Really Know Their Customers? – RetailWire
Discussion Questions
How should retailers approach building a Big Data strategy? What would you add to or stress in the recommendations offered in the EKN study?
Much of the interest in Big Data strategies centers around the opportunities for marketing, merchandising and customer relationship. The challenges here include the nature of the data sources and tools available. Many of the data streams related to these functional areas come from online sources related to multichannel applications. This data tends to come from outside the enterprise, and has to be corralled into a manageable framework for analysis.
Lower hanging fruit for retailers is likely in the areas of supply chain, store operations, and merchandising.
“Big Data” has implications literally everywhere in the retail model, from concept to consumer. While it is important to have a strategy to focus and invest wisely, it is even more important to create a fast test/learn environment. This is an area with the opportunity for real competitive advantage for the early adopters.
Hire experts — people they are unlikely to already have on staff (no offense to those on staff — this is different stuff).
“Big Data” seems to be suffering from the same problem as “the Cloud”. Everyone is talking about it, but no one has clearly defined what it is.
My own presumption is that new “Big Data” is the data gathered from unstructured sources, mashed up and made available in near real-time. The rest of “big data” has always existed and only 2 things kept it from becoming a reality: 1) retailers’ addiction to details and 2) hardware speed.
Hardware speed has improved. Retailers’ addiction to details? It’s hard to tell. Plus the siloed nature of most large retail enterprises makes it challenging to share data across departmental lines. To wit: I think the SKU rationalization projects of 2008-2010 suffered because they rarely looked across categories. Instead, category managers were called upon to play “lifeboat” with their SKUs. Not so effective. Market basket analysis (is that big data? I’m not sure) was not employed frequently enough.
So I do think that every article that refers to “Big Data” should start with a going in premise — “What do we mean by big data?”
I participated in the Time Magazine article, and found it hard to explain Big Data in a way that a layman could understand. It was frustrating for me (and I suspect the writer too). The example he used ended up as a discussion of Video Analytics — a technology that has been around for a decade.
So, my vendor friends, it’s probably not a good idea to pitch “big data in the cloud”. And my retail friends, ask yourself – is your company ready to get out of the weeds of details so that big data will actually mean something to you?
What does Big Data mean? It is taking in more than you can handle because you neither have the platform nor the roadmap to manage it, respond to it and plan using it. I think the article and the survey respondents are right on when they identify the gap in analytic personnel to build predictive as well as trigger based models, to drive key strategic decisions.
At the shop.org conference last week in Denver, analysts indicated that Wall Street is showing great favor (stock price going up) toward retailers who plan and do invest in “IT.” The all-inclusive category that ultimately supports and enables the enterprise to manage all sources of data coming from both traditional (point of sale, web checkout, direct order, call center) sources as well as monitoring/ planning and analysis (inventory, pricing, logistics) and of course social channels.
Big Data is a Big Term. No wonder industry doesn’t understand what it is, let alone how to use it.
This definition from IBM works really well:
“90% of the data in the world today has been created in the last two years alone. This data comes from everywhere: sensors used to gather climate information, posts to social media sites, digital pictures and videos, purchase transaction records, and cell phone GPS signals, to name a few. This data is big data.” (source: http://www-01.ibm.com/software/data/bigdata/)
So the question on the table for marketers is, what information, or information trails, would be most useful in driving business? The answer usually comes in the form of “connecting the dots,” meaning, connecting the social media shares with purchases with those who have seen TV ads. That kind of thing.
Now, for retailers and brands who are interested in making those connections, the best I’ve run across is a new methodology called “cross-media measurement” or “single-source” measurement. Single source measurement is my favorite. Its principle is to follow a single shopper throughout all of her media exposures and tie them to purchasing. That’s pretty neat! Vendors in the space include Nielsen/Catalina and KANTAR Shop.com.
Retailers need big data from a customer experience strategy and should use the capabilities to help enhance the experience by understanding the customer needs.
Big data analytics need to become a retailer’s core competency. Retailers should start to define a big data strategy and then assess whether or not they have the talent on hand to use it. The key point to focus on is customer segmentation, and aligning the best marketing by customer segment.
Developing a Big Data strategy takes a company in the wrong direction. Figure out what big data can be used for or what it can do for you, or how you can use its formation from a 360 degree view of the consumer. Yes, big data is different, but it is still data. The strategy has to be about how you want to use it, not about the data itself.
Thanks for linking this back to the IBM definition, Liz. Let’s remember that Big Data is bigger than retail. It’s a silly buzz term, inappropriately named much like omnichannel marketing (should I really market everywhere? Sometimes I think everyone should be required to take Latin).
So Big Data is “Big” because it includes lots of data that comes from different places other than simply your POS, your traffic counts, or your shopper card. So in reality it is no different than attempting to look outside the walls of your own operation. The forward thinking retailers have always done this. They were the first to try to understand accurate share values.
So Big Data ends up being much more about the how than the what from a definition standpoint. How will I zip this up, compare dissimilar data sources, use it to create actionable insights that can lead to marketing plans utilizing new platforms outside or inside my proprietary walls?
Anyone think the ideal CIO should really end up being a Stats/Software specialist vs. a Hardware specialist? I do.
What is big data? Every day, we create 2.5 quintillion bytes of data — so much that 90% of the data in the world today has been created in the last two years alone. This data comes from everywhere: store POS, posts to social media sites, supply chain and procurement processes, purchase transaction records, and cell phone promotional text messages to name a few. This data is big data. Fully 80+% of this data is unstructured. That creates a mountain of pain for both retailers and CPGs, alike.
I believe that big data is all about “the art of the possible.” Retailers and CPGs can learn to leverage this flood of information to capture shopper insights, and take more of the “gut feel” out of managing their businesses.
Comprehensive tools are available today to help manage this information through deep analytics. This effort, however, is not for the meek. Internal teams should be created to partner with external experts to define a strategy to leverage big data. 1 in 3 business leaders don’t trust the information they use to make decisions. How can you act upon information if you don’t trust it? Establishing trust in big data presents a huge challenge as the variety and number of sources of data grows.
My recommendation is to dive into this area which touches literally every aspect of our businesses. You need not throw your hands up in the air and wait for the world to simplify the solution approach. This is being managed by companies today, large and small.
My business has been all about deriving meaningful insights from large quantities of quantitative data and translating those into tangible ROI metrics and one key lesson I have learned is that brand and retail marketers and merchants have been flooded with data for years. All that data has provided insights, but it is always looking in the rear view mirror.
What marketers and merchants need in this digitally-empowered shopper landscape is recommendations on what to do tomorrow, next week, next quarter. Leveraging data and deriving recommendations based upon predictive science is what marketing and merchandising executives from both sides of the aisle — brand and retailer, will value.
Usually I caution “let the other guy go first” with these technology waves such as Big Data. In this case, even though there are miles and years to go before a complete, comprehensive solution is proven out, I would advise that retailers select an area where they could utilize this technology and begin a project. That’s because there is great insight to be gathered from these data sources. Focus on a single area and see how the results can be applied to the business. Lots to learn, lots to gain in this experimental endeavor.
As always, I like to remind retailers that everything goes back to what their consumers expect from them in each channel. Then they can address how and what Big Data can be used. I understand that many retailers do not have the talent and/or resources in house today, however, I think their are plenty of SaaS partners that can fill this void.
Big data has potential big payoff for retailers, but few ever cover their costs. A lot of data does not mean good data. Too often I see tons of worthless data being stored. It is either at too high a level, too old, or contains multi-meanings.
There are 3 steps in working with big data. 1. General analysis and findings, which answers the question of who are the customers. 2. Answers questions; example: who buys X product and when? 3. Evaluate an action, example a 9 cent price reduction produced X% sales increase.
The most missed big data issue is trends. The Great Recession changed some buying patterns. Who changed, and what are they buying now? There are both short- and long-term trends which should be monitored.
As many are pointing out this morning, “Big Data” is an overused and often misunderstood term. Paula and Liz provided some good definitions in their comments.
Like any tech-centric project, the key for retailers’ Big Data projects will be to define the business objectives in advance before picking the tool; but it’s important that the facilitator of that discussion is someone who understands the capabilities that are out there.
Once the objectives are defined and the project is underway, broad adoption by business users (District Managers, Marketers, Merchants, etc.) has to be a key priority. The data visualization tools that have recently come to market can go a long way toward making the output of a big data initiative intelligible and actionable for front-line business decision makers.
Their Big Data strategy should drive or support the overall strategy. This may sound simple, but it’s a bit harder than it sounds for a number of reasons:
1. retailers have historically not been that great at strategy but focused on operations and the day-to-day
2. the digital/Big Data changes are happening simultaneously across the entire retail landscape and a retailer likely can’t pursue all fronts at the same time
3. the changes are blurring the retail boundaries. Retailers will need intermediaries to bridge to other datasets without creating privacy/data security risks
4. the landscape is changing fast and making bets on external capabilities could be expensive (e.g. My Space, Second Life, Groupon etc.)
5. CIOs are more focused on technology for operations than the “I” for information that their title suggests. Their teams are typically organized and skilled accordingly too.
These are not the only reasons, but highlight some of the challenges….
This should be a key focus of any retailer, large or small. Data mining, and using this data should be part of the marketing efforts of all retailers. We have this information, yet more often than not, use little or none of it, and yet try to find this information later on during our marketing and retailing efforts. Such a waste when there are answers to so many key retailing issues right at our fingertips, if we would just take the time to mine this information.
“Big data” is a big deal, but how you approach the opportunity can greatly impact the outcome and overall value of the exercise. I agree with the study; pick three key areas that will have the greatest impact on the business and focus on those first. I find it is most important to understand the questions you are looking to answer before you decide what and how you will collect any data.
The answer is that the focus needs to be much more on ROI than it is so far. The technical infrastructure is advancing to be sure, but who cares if I can’t make more money. For that, you need the right analytics.
Any data set will have spurious correlations. The “Bigger” the data, the more spurious correlations will appear. The key is to have a method for sorting out the signal from the noise, identifying what actions actually drive incremental sales, specific reduced costs, improvements in satisfaction, etc.
Big Data will make a difference in the world, but only the real analytical leaders will get to the right insights (not just useless patterns) anytime soon.
The same organizations struggling with Big Data are the same ones who have been collecting mass volumes of consumer data for decades and have under-utilized the asset.
Organizations should start the conversation about Big Data by asking their colleagues: what do we collect now? What do we do with it? What potential exists if we put our existing data to use?
By making this internal assessment, the organization can create processes and policies about data use and achieve tangible business results with the existing asset. From that foundation, the organization can add data from new digital channels and create incremental value based on this foundation.