I’m not sure who is complaining about Big Data other than the processing and analyzing of data has always been a formidable task for most retailers given their general aversion to investing in talent and systems to get the most out of the data they generate. But the larger problem of data overload that Bill Bittner speaks to is real. Having dealt with these issues first hand, I can also assert that without a C-level strategist who is empowered to effect pricing, merchandising, marketing, and even advertising, data analysis and resulting applications are destined to be more ad hoc than strategic and thus lead to inconsistent results. Once a customer data plan is in place, focus can be trained on specific categories, customer segments and even stores, that is necessary to target a limited content to the areas of highest return. Having a plan and strategy also will be a positive signal to CPG brands who are looking for retail partners they can trust long pull to use data to drive their mutual interests. “New… Read more »

Having supplied brands and retailers with recommendations and guidance based upon quantitative data and analytics, I’ve found that they are drawn to the absolute nature of data but often lack the commitment and cultural collaboration to respond to what the data is telling them. People want ‘actionable data’ but seem to be reticent to actually respond to the required action that the insights are suggesting. The insights often require a cross-functional (departmental) response and that becomes difficult to actually execute.

There is certainly an interest to value data and BI that supports an existing agenda or initiative, but when the insights suggest a different compass heading, the head winds that would be encountered to facilitate a change are often too difficult to overcome. Retailers tend to be focused on operational issues and look for the ‘quick fix’ on existing processes and workflows rather than designing innovative (and often disrupting) strategies based upon quantitative data and customer/business intelligence.

We are not in the business of generating or selling data. We are in the business of helping people figure out what to do with data — and that does not mean “tools.”

Bill hits the nail on the head. Databases are like dungeons. If you don’t know what you are looking for before you go in there and don’t have a torch to light your way, you will never come out.

Clearly the problem isn’t technology. And frankly this isn’t “big data” either, in fact in the grand scheme of things it’s rather miniscule. “Big data” is what geneticists are going through when they have to wait for over a year for a search of their data to be completed because it’s so massive. The most dangerous part of a swimming pool isn’t the deep end. It’s the narrow section between the shallow and the deep where you don’t know whether to stand or swim. That, it seems to me is where retail is when it comes to data and perhaps technology as a whole. ‘Back in the day’ a retailer could ‘feel’ what was happening, when and how to change an approach to a customer, what products would work and so on. To do that retail had to be in your blood. Now we’ve complicated the customer experience to such a degree (most of our ‘insights’ we just made up and have little validation for) that we’re looking to technology to make real sense of… Read more »

Data mining and needing a true use for the information, let alone knowing how to use the information, are the key issues. At the heart of all of this is that retailers are capturing data, but truly have no use for much of the information. Data mining is a solution looking for a problem. Retailers KNOW that they have to prevent OOSs, give better customer service, and keep their stores clean, clear and easy to navigate for their customers. Training their staff, and maintaining low prices are the other side of the coin (sometimes). However, these key basics are still not addressed enough, let alone what to do with the data that their POS system captures every minute of every day. The expertise at retail is the customer’s retail experience, NOT how to mine data to squeeze every bit of information about a particular product, category or time period. This is why Data Mining and use of this information continues to be ignored….

What a timely discussion after a great NRF show in NYC this week. I started working with retailer data (grocery) 15+ years ago and even then retailers — and frankly, manufacturers — were complaining about too much data. It’s not about the data and how much or little you have, it’s about the questions you are laser focused on answering that count. People still think that if you collect all the data and run it through a BI solution, a magical answer will come out the other side that will make you more successful. I sat through a presentation on Tuesday presented by the CFO of The Vitamin Shoppe and the President and CEO of 4R Systems. In one hour they took us through their process of dramatically improving store performance through the use of data. They started with a laser focused question and worked backwards from there, using the data available to create power answers that they then acted on and benefited from. It’s not necessarily that 4R Systems has some magical solution, it’s… Read more »

The powers that be in retail intuitively want big data and all that it brings. However, the reality is that they can’t execute against it. It tells them that the way they have been operating (from their gut) for decades is wrong. Big data is going to tell them to change the way they address customer acquisition, the way they handle inventory and the way they project their sales.

The challenge is not to collect data (retailers have plenty of data to mine) nor even to find a software to make sense of it. Retailers must work in partnership with their software vendors to go beyond a laundry list of features and benefits and produce the reports that really matter to them. What matters the most to retailers is neither the quantity of data nor the software features, but the access to actionable reports that retailers could use as coaching tools. Very few software vendors actually take their clients from viewing to doing.

Big data tend to be anecdotal talking points among people who are not in the trenches. After doing some work in the trenches in retail, I realize that “small data” is very important.

I believe many BI companies are trying to bring slow corporate BI practices to fluid environments such as retailing. Retailers are complaining because KPIs and dashboards do nothing for them when the weather forecast is predicting rain for the next several days.

Retailing needs real-time data in small chunks at all levels, not past 12 year forecasts and trends. I would argue from my experience, retailers need more “predictive” data that can monitor real-time data and detect and respond to patterns in real time.

Bill: Great insights, and I think you hit the nail on the head: retailers are complaining about big data because they don’t know what they want from it. Big data won’t help you if you don’t have a strategy already.

The other problem is the age-old “in-house” syndrome. Too many retailers want to manage these massive data sets internally, when they are clearly beyond internal capabilities.

Big data is clearly the future of successful retail, but effective use will require new skills, a willingness to outsource, and a clear strategy.

Know your path and follow it. Capture info and it goes into a repository that neither has the logic in place, nor the tools to empower the merchandising and/or the marketing side of the house, and all you are doing is creating “big data” with no clue. Consultants who have a process for asking the right questions to lead the enterprise near- and long-term can help. Customer intelligence at work is a mantra to follow — if it’s collected, there better be a plan on why you are collecting it and what you are going to do with it. Old habits die hard and some on the merchandise planning and allocation side of the house have their long-standing methodologies to drive their decisions, or they haven’t seen a dependable/usable solution to support a change. Lots of vendors selling solutions to overcome this complaint. What are they missing? My impression is that it is a lack of strategic/tactical application informed from the data. The tools are smart, but as shared you need in-house resources or hire… Read more »

Okay, I sort of think we’re missing the point. The value of “big data” is getting actionable insights at a granular level QUICKLY into the hands of decision-makers (NOT score-keepers). That has never been possible before. Moore’s law, wide data pipes to stores, and mobility has helped make that possible.

So what retailers need may be the same, but the speed at which they can get it has changed – dramatically. That’s what big data is all about. It’s a short-cut for “actionable insights at granular levels delivered in near-real time into the hands of decision-makers.”

That’s a very new story, not an old one at all.

Retailers who complain about the challenges in translating data into actionable insights are simply not prioritizing. It’s now easier and less expensive than ever to manage Big Data but retailers consistently under invest and shift priorities.

That said, it’s still not easy as it takes people, tools and discipline that is generally not resident within most retail organizations. The good news is that this is changing and every day we see more retailers investing in this area. Whether it’s through an innovation group or marketing or even IT, it’s happening and will continue to happen at better retail organizations.

It’s very clear if you read the quarterly earnings reports and listen to the conference calls, the retailers that understand Big Data and focus on customers, best exemplified by Amazon, are the ones that are thriving and putting the others out of business.

Across industries, executives are swamped with data. Many companies extract the needed information. The big problem is the look and feel of what is reported. It’s up to the buyer and user to determine what they need and how best to report it for action. Accepting the typical template and dashboard can lead to feeling bombarded rather than supported. Business intelligence can only be activated when those examining it can easily determine what they need to know. A little up front planning as to how to organize and show the output can make the difference between data and insights.

Heck, retailers have never been able to manage data, even before they didn’t have too much to manage. The inability of retailers to manage data resulted in laws being passed in the 50s and 60s that fined retailers heavily for running out of merchandise that was advertised via a sales flyer or radio/TV. This law was lifted in the 80s after some very heavy and expensive lobbying. The fact is that retailers often just don’t have a clue. I have two local grocers (chains). Every year during the Christmas Season, one of these retailers runs out of stock on holiday baking supplies, the other does not. This has happened every year for the last ten years. Now the OOS retailer has been bankrupt twice and has just recently been sold, so I expect the problem to disappear. However, that will ultimately depend on who in the old organization the new owner retains. But this multimillion dollar retailer had tons of supply chain information and consumer complaints, but chose not to act on the information. If… Read more »

Tools are an enabler, but are not the whole solution, and in fact may in some cases make life harder for the retailer. There are broader organization fundamentals that need to be in place to be successful and many are at odds with the way retailers are used to working. If you are interested in some perspectives on these, here they are… (1) A clear strategy that the organization is truly behind (many retailers run on tactics and financial metrics). This allows the business to know what direction to head in, which questions to ask and how to apply the answer consistently across the business over time. (2) An organizational design and readiness that can use and apply the insight. For instance, people with the right skills; organized to transcend the politics; processes that incorporate the use of the answers from the data; and incentives that focus on the right behaviors and insights (not just short term, one-dimensional product or category focused financials). (3) The capabilities to turn the raw data into useful data. It’s… Read more »

My experience is skewed in FMCG, especially grocery, which I’ll speak for here. Let’s first describe “Big Data” from a retailer’s perspective. I suggest it is the detailed transaction history from their POS systems, for those with online channels, online transactions included. There continues to be intense retailer interest, due to proven returns, in shopper centric marketing. Derivations of transaction data inform assortment, pricing, promotion, and advertising decisions with increasing localization and also support highly relevant individual customer communication. Software “tools,” including BI systems, are only enablers of this movement. Frankly, many BI platforms are not especially good at the necessary transactional analytics. The central issue is the effective application of the tools, within a retailer that is fully committed to a shopper marketing culture. Imagine bringing your sick automobile to the dealer and being given an engine analyzer and a chest full of tools with, perhaps, some encouraging words. This is the plight many retail exec’s feel they face. The Big Data problem is solved when domain knowledgeable expertise is applied with domain-specific software… Read more »

This one is simple. The grocery industry as a whole has inadequately invested in systems and personnel since the first round of scanners hit stores. Many are currently saddled with legacy systems that need to remain in place for purely financial reasons.

To make matters worse, we are now living through a period where a popular strategy is to delay capital investments. So, instead of moving into this decade’s opportunities to use new technologies in more forward looking applications, many companies are content to wait just a bit longer. Heck, we don’t even really forecast in this industry — we scenario plan. Not too many math research degrees in the halls of grocery companies.

Data is a wonderful thing; it can speak to us with great nuance, but you have to be able to extract from that data relevant information and actionable knowledge, and then you have to be able to execute on that knowledge. I don’t think many retailers can really do either, and I think they know it. The challenge with extracting relevant information and actionable knowledge is that what’s relevant is always shifting and moving. This is where human experience and judgment is critical. In order to get the right answer you have to know what the right question is. It’s not necessarily pre-programmable. Further, the questions that senior retail executives are primarily interested in is how can I reduce my costs. I don’t really think they see data as a way of increasing sales, they see it as a way of containing costs and keeping up with the competition. As a result, they simply aren’t investing in the skill sets necessary to consistently convert data into relevant information and actionable knowledge. That leads to the… Read more »

The answer is simple, but execution is always the challenge. In order to leverage the data and transform it into something actionable, a retailer needs to do one of two things: 1. Hire the skilled personnel who understand the business and the tools needed to extract actionable information, or 2. Hire a partner who can do the same.

The problem is that the data so far outweighs intuition and ‘innate retail intelligence’ (i.e. gut) within retail orgs now that most people reading the data don’t know what to do with it. As in any research, there’s always a qualitative factor…why are the numbers saying that? What should I ask these numbers for? Etc. Who would know that now? The CEO that was a CFO?

Goes back to another retail dilemma; are we hiring the right people? Are we hiring too many scientists and too few merchants? I personally think, yes, the latter is true.

There’s a big difference between data and knowledge. Data are necessary to take store management out of the realm of guesswork and into that of true, fact-based decision making. But data points on their own are highly difficult to work with, especially when we’re talking about the complex, nuanced environments we’re seeking to optimize. Our customers have achieved great success by using a solution that takes the huge quantity of data available and converts them into views and reports that yield actionable insight. Our solution measures more than 9000 individual data points on the average store visit. Nobody wants to examine 9000 data points per visit. Instead they want to see correlations, comparisons, and models that help them understand the true behavior in the store. They want to see heat maps of traffic and how those maps change over time. They want to see the cyclicality of the day, week, and year and how that compares to staff schedule. They want to see how all of these things correspond to actual sales at the register.… Read more »

The article missed the point. What Teradata, Micro Strategy, etc, offer as analytics solutions have been around for some time. The advent of Big Data is also not new, albeit with a different name — item level data, RFID data etc., Application logs have been in existence for ages. The existing analytics apps as-they-are cannot address these volumes of data, their asynchronicity and demand for low latency.