Does “traditional” market research have a role in the future of insight creation?
Greenbook has just released the latest version of its Research Industry Trends (GRIT 14) survey of market research buyers and sellers. Among other findings, the report reveals trends in client organizations’ use of “new” research technologies such as online communities (down), mobile surveys (down), social media analytics (up), text analytics (flat), and big data analytics (up). The technologies that are trending up have one thing in common: they do not require direct active interrogation of consumers. Almost all of our “traditional” market research techniques, on the other hand, require some kind of dialogue with the consumer, whether or not it takes place in real time.
A couple of days before GRIT-14 was released, Knowledge at Wharton (K@W) published an article titled “Finding a Place for Market Research on a Big Data, Tech-enabled World.” Key contributors to this article include Pete Fader and Eric Bradlow, Wharton professors who co-direct the Wharton Customer Analytics Initiative, as well as Shawndra Hill, professor of operations and information management, and marketing professor Barbara Kahn. Fader and his frequent collaborator, Bruce Hardie, are best known for developing models of consumer behavior based on purchase transaction and other types of passively-captured data (such as in-store movement). Their work has yielded important insights into consumer behavior.
In the K@W article, Fader suggests that many firms have moved away from what we think of as market research, often preferring to “just try stuff and see what works.” This shift may be due, in part, to the outsized influence and successes at Apple of Steve Jobs, who famously disdained market research. No research at all is one manifestation of the shift away from traditional market research; the increase in analysis of all types of passively acquired consumer data, whether “big” or “small,” is another.
Part of the problem might be that we’ve lost clarity on what we mean by “market research.” I agree with Fader that market research is all about assessing consumer preferences and attitudes to understand what they are thinking and how they are making decisions. This is the core of “consumer insight” and has always depended on our ability to match what customers are doing with what they are thinking. The passive technologies—text analysis of social media, social media network analysis, and data mining (for all kinds of “big” data) give us only part of the equation. Mining transaction or clickstream data for patterns can tell us what they are doing but not why. Text analysis might give us some vague glimpse of the “why” but not relate that vague “why” to actual behavior. Curiously, at the same time we are seeing a push towards looking for insights in passively captured data we see the rising influence of behavioral economics in market research. Behavioral economics is all about psychological processes, and these are hard to divine from the analysis of social media and transaction data.
The market research value chain consists of four activities with the potential to add value:
- Framing the question that needs to be answered (you’d be surprised how often firms fail to do a thorough job of identifying the real question).
- Gathering data or facts from and about customers or prospects that the firm does not otherwise have access to.
- Summarizing and analyzing those facts.
- Interpreting the facts to yield insight into consumer motivations and behavior.
As long as there are facts that cannot be obtained without directly engaging consumers in a conversation (whether by informal or highly structured means) there will be a place for traditional market research. At the same time, the new data sources give us much more to work with, particularly with respect to finding patterns of behavior that were previously invisible to us.
So there is definitely a role for “traditional” market research in the future of insight generation, as well as roles for social media analysis and big data.
– David Bakken, PhD, Chief Insight Officer