Photo: Getty Images/kupicoo
What are the hurdles to becoming data-driven?
A new survey of chief data or analytics officers finds that only 26.5 percent believe they have created a data-driven organization.
NewVantage Partners’ “Data and AI Leadership Executive Survey” found less than half (47.4 percent) replying that they were competing on data and analytics. Only 39.7 percent said they were managing data as an enterprise business asset, and just 19.3 percent have established a data culture.
For the fourth consecutive year, over 90 percent of executives (91.9 percent in 2022) point to culture as the greatest impediment to achieving this business outcome. Only 8.1 percent cite technology limitations as the primary impediment.
A new survey among CIOs, chief data officers and VPs in data-oriented positions from cloud software provider Domo found the top cultural or attitudinal challenge for using data effectively is that too few people across most organizations understand how to access useful data or use it to drive meaningful decisions.
Domo’s study pointed out that the two most common explanations for the significant data decision gap were BI tools being disconnected from business processes and a lack of training.
A survey of c-level executives from last fall from the credit reporting agency Experian found a leading 44 percent of respondents say that a key indicator of being a data-driven organization is a high level of trust in the quality of data, followed in the top-three by the presence of easy to use data management tools (40 percent) and speedy, flexible access to data that can be scaled as needed (39 percent).
In a column for the Harvard Business Review, Randy Bean, founder of NewVantage Partners, said organizations can accelerate the journey to becoming data-driven by focusing on three principles:
- Think different: “There is no shortage of analytic algorithms. These need to be matched by critical thinking, human judgment and a view to creative innovation.”
- Fail fast, learn faster: “Companies that are prepared for faster iterative learning — fail fast, learn faster — will gain insight and knowledge before their competitors.”
- Focus on the long-term: “Becoming data-driven is a process.”
- NewVantage Partners Releases 2022 Data And AI Executive Survey – NewVantage Partners/Business Wire
- Data and AI Leadership Executive Survey 2022 – NewVantage Partners
- Why Becoming a Data-Driven Organization Is So Hard – Harvard Business Review
- New Domo Study Shows the Data Decision Gap is Holding Business Back – Domo
- 2022 Global data management research – Experian
Discussion Questions
DISCUSSION QUESTIONS: What advice do you have for retailers and brands looking to become data-driven organizations? Is the primary impediment company culture or is it more about data-quality, lack of training, technology or some other process issue?
Poll
Loading …BrainTrust
Don’t confuse data gathering organizations with data-driven organizations. Collecting data is easy; turning it into insights that can be used to deliver different/better outcomes is hard. Culture is certainly part of the impediment, but I’d say lack of will is bigger. Most retailers know what they should be doing, but they simply lack the will to do it and commit to it. All the other factors like data quality, training and technology play a role, but without a commitment to using data in a meaningful way, survey results like these will persist.
I agree fully, Mark, but don’t you think lack of will is part of culture?
Perhaps, Gene. It may just be semantics, but I was trying to focus to the element of “culture” that may be the root cause. To say the problem is “culture” is vague.
Yes, culture can be very vague and often an excuse rather than a solution. I agree with you.
It’s interesting to read that few senior data-management officers feel that their own companies are sufficiently data-driven. (If you asked the company CEOs, they would probably give a much different answer although they aren’t as close to the ground.)
Part of the data managers’ response may be self-serving, since they would argue for more spending in every budget cycle, but they have a point. Are companies maximizing data science to target the right customers, to offer more regionally tailored assortments, or to bend the curve of supply chain costs?
All that being said, CEOs need to find a cultural balance between “data-driven” and the kind of risk-taking that has always characterized the great retailers. The trick is to create an atmosphere where both data and “gut instinct” can thrive together.
Unified commerce is the single most important aspect of driving analytics in companies, especially retailers where data has forever been siloed. The complexity of today’s multichannel process including the added areas of social commerce and mobile commerce makes it a challenge to effectively draw insights across structured and non-structured data. The primary impediment for retailers using data-driven decisions is culture — just like the culture of innovation, there should be a culture of using analytics to drive decisions combined with human guidance and intuition.
One doesn’t create culture in a single step. It is an ongoing process, made by many incremental discussions and actions. To become enculturated as a data-driven organization requires developing the means to collect and analyze the data, which is relatively easy. It also requires a pervasive mindset of how to internalize, analyze, and synthesize data into meaningful conclusions, which is much more difficult. It is hard to convince people to trust the data when their emotions tells them otherwise.
Mark Ryski is correct that a will to use the data is critical – but I think that organizations who want to become data-driven need to ask themselves, why? Data is useless – information for business decisions is priceless. Too many companies go for the former and not the latter. First understand what you want data for, then determine the best way to get that data, then test ways of putting the information that is analyzed into the hands of decision makers.
Changing corporate cultures is never easy, and rarely is it quick. Becoming data-driven requires a steadfast commitment to forecasting, monitoring and measuring the business through data-driven analytics. Leadership must insist upon data-driven analysis of decisions at every level of the enterprise – even if some of the recommendations run counter to the data. To get to that point, leadership must learn to trust the data. That responsibility falls to vendors to present verifiable and consistent results that earn the trust and confidence of client leadership teams.
To paraphrase a line from Animal House, “having poor training, poor process and poor data quality are no way to go through life.” The Holy Grail of success is people, process, and technology. Most organization have or can acquire the technology but they skimp on the people and process parts which inhibits success. You must focus on all three components to be successful. Bad process creates bad data and lost confidence in the technology.
This is really about the difference between data and information. Here’s an analogy: There’s lots of data — real-time data — needed for air traffic control. But a controller needs to see blips with altitude and speed to keep planes from crashing into each other. That’s the information that prevents disaster. It also lines up with the Experian findings about trust and ease of use regarding data.
I am stuck at a far simpler question: Is there any strategic advantage for a retailer becoming “data driven”? The mere term speaks of giving up human judgement and enslaving the company to the average – one can be no more than what the data reveals.
There is also significant strategic risk. Should a retailer become data driven, it is unlikely to discover important things and spend its time on them, rather it spends its time chasing whispers of small differences among customers.
Data is important and we should all manage with a good eye on data – but also a good eye on customer research, buyer instincts, and a flair for the store. Data is merely one element and not nearly as valuable as it is sold to us.
Well said — there is a clear bias towards believing that data matters. Sometimes it can be very useful, other times it overshadows good retailing and merchandising. In the worst case, it becomes something to hide behind.
To evolve into a data-driven company, executive sponsorship and a change management plan are essential. Companies can no longer afford to make decisions based on guesses or “what we’ve always done” — they need data.
To earn workers’ support, the change management plan needs clear, continuous communications about what’s new, and how the company and employees benefit. KPIs tied to processes, systems integration, training and data integrity are also key success factors.
Having worked with countless retailers and consumer brands, the biggest challenge that they tend to have is arriving at a single source of truth. The primary reason that they tend to become less data-driven is because data fidelity and reconciliation is time-consuming and can lead to frustration especially for speedy decision-making. The cultural change required is convincing decision-makers that the investment needed in systems, tools, people, and process is worth the effort in the long-run.
Data-driven? More like “data-deficient.” The results are not surprising, as prior to the pandemic most organizations talked a good game that they were data-driven but, in reality, it was on the back burner. “Yes, it’s important, but we have more important things to do now” was a popular refrain. Well, “now” is here and retailers and brands are still struggling.
There are five key pillars for data-driven organization success: 1.) Organizational preparedness, 2.) tech readiness, 3.) data acquisition and flow, 4.) stakeholder involvement, and 5.) reporting and advanced customer analytics. Most of the time, the primary impediment to data-driven companies is culture and process.
Even in the era before today’s data availability, it was not unusual for a CEO or marketing director to ask for research from a competent organization, only to throw it out because they didn’t believe it.
Retailing, especially fashion and advertising, has always been based on someone’s gut feel. “I know more than this data shows.”
Randy Bean’s column prescribes the proper implementation well. The challenge is, as the discussion suggests, culture. Which starts at the very top. Unless a leader accepts contradictions to what they think, data, no matter how many millions companies spend, will never be taken seriously.
Two key points. First, culture does play an important role and it starts at the top. Company leaders must fully embrace the notion that one of the greatest assets a company owns is its data. But waving the wand and saying we’re now “data-driven” is not enough. Companies must invest in people skills so associates can not only understand, but also capitalize on the technical aspects of data science. Moreover, companies need to invest in the right tools and enterprise-grade analytic platforms, enabling massive compute capability, which will return insights in seconds/minutes vs. hours/days (think of crunching 300 terabytes of data in less than 10 seconds).
Second, CEOs must demand more value creation out of the CIO/CTO/CDO office. Instead of asking for cost reductions of 10 percent each year, they must flip this and invest more dollars in analytics, people skills and technologies. Alarmingly, 80 percent of algorithms are NOT operationalized within companies, contributing to a massive gap in the number of insights that could be uncovered vs. what actually is. This translates into huge upside, but it won’t be realized if CEOs don’t mandate the use of analytics that underpin nearly every company decision.
Retail has always been awash with data but trying to make meaningful sense out of it has been evasive for most. New technology is rapidly providing greater insight, making much better use of data for operational and more strategic advances. However it is imperative is that C-level executives know how to ask the right questions to uncover the information that they need to drive the right strategy for their business.
There are no magic plug and play solutions that will give a company its strategic direction. Data is what feeds the organization and provides the insight with which to make the right decisions. Interestingly, many companies have a culture problem because people think they will be put out of work if data analytics take over. The reverse is true. Data driven companies require bright people who can interrogate the data to drive the business forward.
There are two major hurdles to becoming a truly data driven company. First is being able to sort through the plethora of data to find the insights that really matter for the business. This speaks to data quality in many cases. There’s also an issue about confusing things that are “neat” in the data with data that is relevant and actionable. The other issue I see is willingness to accept insights that are contrary to leadership’s ideas about strategy. I’ve worked with many retailers who quickly disregard data that goes against their “gut” feel. While I do believe that there is a place for intuition and product knowledge, today there’s no excuse for dismissing insights out of hand because we think we know better.
The hurdle is both cultural and technical. Many organizations think that BI tools can be used by all and for all use cases. The reality is that many struggle to use the typical BI tools like Tableau and Power BI. They are powerful tools in the hands of those that have been fully trained. They are not, however, for most employees and certainly not for all use cases. The culture changes more effectively when you provide the majority of the organization tools that are easy to use – that work with the skill sets most business users have and in a format that they feel comfortable using, to create data-driven content that can be used to make decisions without having to wait for the BI team to get to it. Our industry moves too fast to have to wait for another team to go through the data-to-insights process.
You can indeed change the culture of the organization to be more data driven but you have to provide solutions that will be adopted by the majority of users in the organization instead of just a few. In other words, you need both the sophisticated BI tools and the ones that users are already comfortable with. Most business users have decent skill sets in PowerPoint and Excel, especially in CPG and retail. Provide an easy way to access the right data for each use case, and business user-friendly tools to prep the data for presentations and reports and you’ll have an organization that is much more data savvy and more comfortable generating insights from the data. That will be the game changer, and it’s been proven time and again.
The starting point for becoming a data-driven organization is to understand where you are today. First, do an audit of how data is used and ask your people the amount of time and effort it takes to make decisions and do analytics. Many times your most talented analytic professionals are spending more than one-third of their time in data collection and prep. You are not enabling data-driven decision-making if your people have to go through hoops to bring data together for each analytics project.
Data integration, transparent processes, and data governance are the keys to help enable your teams to be data-driven. Data management tools can provide the foundation for a trustworthy set of business data that can be used operationally, fuel your commerce engine, and also analytics. Enabling individual and team success is key to becoming more data-driven.
It’s a bit of “yeah, we tried it and it didn’t work” and yes, that is probably true. Today, however, dashboards, insights and actions are much more digestible for all levels and cultures. The hurdle is to build trust that using data will not only improve the organization but the department and individual’s work as well.
Data by itself is useless. The strong retailer needs data to make decisions, not just gather data or follow it blindly, so the assumptions and circumstances affecting data and potential consequences have to be considered as well. As with any good decision making, both quantitative and qualitative information have to be taken into account. You wouldn’t buy a fridge that has all the features you need but can’t fit in your kitchen even at a discounted price. Data with the right assumptions and context provides appropriate decision making. Step two is building the data to be credible, repeatable for making decisions, and validated to show explicit results, which can be cultural, but depends on the organization. Data doesn’t drive decisions, but it can lead execs to make better decisions- and for that process, culture, context and data understanding is crucial.
The data is available. The key is to know which data is important and useful. Less is better. Once you have the data, the next big hurdle is to react and take action.
As simple as it may seem, to become a data-driven organization, you have to empower (and require) employees to back up *all* decisions with data. Sometimes people bury their heads in preconceived notions or old “files” so making the shift to data-first can bring some friction to an organization. Lastly, it is critically important for retailers to leverage technology tools that have strong built-in reporting, BI, machine learning, and AI — so all of the data is woven into the day-to-day processes and not just reserved for a small subset.
Being data-driven is not about the data. It’s about the willingness and curiosity to learn from that data. It’s about the will to act on those learnings. And it’s about the ability/skills/tools to enable that will. Data + willingness + will + skills is a rare combination of attributes to find in one company, as the market reminds us all too frequently.
This really resonates with me, especially the word “curiosity.” Even for companies whose entire business model is the collection, organization, and distribution of data, there must be always be an emphasis on “why.” Why do we want to know this, why do we want to share this, why is this behaving the way it does? Being innately curious about one’s business and its data is part and parcel with being a great entrepreneur. That human drive is then the basis for the culture of the organization.
In my experience, I have clearly seen that the biggest challenge to becoming data driven is change management. The implications of moving to a data driven management approach include data transparency and democratization, which highlights successes and failures in incontrovertible ways. If the culture is not built to be excepting of failure as well as success, then moving to a data driven approach is perceived as very risky.
The greatest success has come with strong management support of the “new way,” clear definable metrics for success of the initiative and lots of small wins that begin to build momentum. A hands-on approach by the analytics team, focusing on the business problems and analytic approaches to help business managers take action, help to add more steam to the data-driven business “engine.”
It amazes and annoys me that retailers and brands have not taken advantage of technologies that have been readily available for years to drive decisions throughout the organization. It’s not so much culture, but a tangible plan to collect data and turn it into insights. I’ve been at the forefront of the analytics movement for years, and the best recipe for success is to think big and work small. Once there is an overall strategy, analysts must have identify tangible project(s), team with like-minded business sponsors and promote successes internally. This approach will grow acceptance and momentum.