Why Big Data Underperforms at Innovation – Part 1
The American Market Association recently estimated that companies will spend approximately $92 billion on big data by 2026. But is big data delivering the return on investment in innovation that leaders want?
Forrester reports that only “29 percent of firms are successful at connecting analytics to action.” Although companies are gaining more information about their customers than ever before, actionable insights that can drive innovation remain elusive. Evidently, companies are overfed on data but undernourished with actionable insights. Why? Because big data reveals correlations, not causality.
The data being captured is the wrong data for innovation. Up until now, data analytics has not been able to determine why customers buy the products and services they do. An insight is most actionable when it reveals causality or why people buy a product, i.e., the jobs customers are trying to get done when “hiring” your offering in a specific circumstance. Until companies learn how to capture the jobs customers are trying to get done with their offerings — functionally, emotionally, and socially — they will continue the trend of high innovation failure rates.
Although companies are capturing vast amounts of real-time and historical data about their customers, it is primarily revealing trends and correlations. While this is extremely helpful for creating extensions of current offerings, it is very limited for creating new-to-the-world offerings.
For example, big data can easily reveal that more people are buying organic foods. For a food company, this trend suggests that there is an opportunity to create new organic versions of their current food products to extend their product line. But it does not reveal how to create a distinctive new food category, because the trend line does not explain why people are buying more organic foods. We have to dig deeper to understand the cause.
In talking with target customers to discover why they buy organic food, we might learn that they want to:
- Prevent disease, e.g., cancer, heart disease, etc.
- Manage chronic illness, e.g., arthritis, asthma, etc.
- Slow aging
- Etc.
Knowing which of these jobs is important and unsatisfied (which may require surveying a statistically valid sample of target customers to get such ratings) enables food companies to develop new organic products that customers will buy.
For example, the food company that knows that “slowing aging” is an important unsatisfied job for 72 percent of people who buy organic foods can now create products that are not just organic, but that deliver specific nutrients that slow aging better than other products. With this type of information, food companies can create unique offerings that address every one of the important unsatisfied jobs customers want to accomplish. These new foods become the delivery system to help customers get their health jobs done better. This enables companies to create superior offerings and establish a valued and unique position in their markets, which is the essence of good strategy.
The efficacy of this approach is demonstrated by PepsiCo, General Mills, Kraft Heinz and many other food companies that have been pouring money into creating healthier products, not simply organic products. Most recently Nestle Foods hired a healthcare industry veteran as its new CEO to invigorate its medical foods business.
Leaders who want to turn innovation and growth into a predictable business process must learn how to discover the jobs their customers are trying to accomplish through skilled interviews with customers, and stop relying on big data. Perhaps one day artificial intelligence will enable big data to identify why customers buy what they do, but until then, go get the structured data you need directly from target customers. Stop trying to hobble together data that is inherently inadequate for the job of innovation. Correlation is not enough.
(A version of this article first appeared in The Business Journals, February 28, 2017.)
