July 30, 2013
BrainTrust Query: Can Marketing Research Borrow from Behavioral Economics?
Through a special arrangement, presented here for discussion is an excerpt of a current article from the Joel Rubinson on Marketing Research blog.
It is important that we marketing researchers become like behavioral economists and study how people make choices, not just the choices they make.
Choices often have three dimensions: what media to seek out for advice, what to buy, and what retailer to buy from. Each choice has its own path that is intertwined and critical to understand.
The fourth dimension — one we researchers think of too infrequently — is the choice of the answer respondents give to a survey question. Survey-taking is chock full of decision-making because people are not opening their brain like it’s "a container" and just letting truthful answers pour out of their heads. They are reconstructing memories and opinions in the context of their current mental state, how the question is framed and asked, and how the preceding parts of the survey have brought a respondent to the next question.
Here are three smart steps to put a little behavioral economics into the process:
1. Nudge the Respondent
Nudge: Improving Decisions About Health, Wealth, and Happiness (2008), by Richard H. Thaler and Cass R. Sunstein, is all about the idea that there is no neutral way to frame choices: "simply by rearranging the cafeteria, Carolyn was able to increase or decrease the consumption of many food items by as much as 25 percent."
Take it to the next step: We need to nudge respondents to get the most accurate self-reporting when compared to known behavioral data, like brand penetration levels.
2. Heat Up the Respondent
We tend to study preferences at times that are divorced from a respondent being in a need state. Noted behavioral economist George Loewenstein describes his research on cold-hot empathy gaps as follows (Loewenstein, Read and Baumeister; 2003): "when people are in a cold state — i.e., not hungry, sexually aroused, in pain, angry, etc. — they underestimate the impact of such ‘visceral’ (hot) states on their own future behavior."
Current concept testing does not do enough to "put people in the mood," especially if the idea is innovative and might create its own category.
3. Create Social Contracts
Dan Ariely’s Predictably Irrational, Revised and Expanded Edition: The Hidden Forces That Shape Our Decisions (2009) raises an important insight about social vs. monetary contracts:
"… they studied a day care center in Israel to determine whether imposing a fine on parents who arrived late to pick up their children was a useful deterrent. … [They]concluded that the fine didn’t work well. … Why? Before the fine was introduced, the teachers and parents had a social contract, with social norms about being late. Thus, if parents were late … their guilt compelled them to be more prompt in picking up their kids in the future. But once the fine was imposed … they frequently chose to be late."
The ARF Foundations of Quality Research program I helped to direct proved that those who are motivated by a social contract (i.e., "Giving my opinion is the right thing to do") rather than receiving cash incentives led to more diligent survey taking behavior.
BrainTrust
Recent Discussions
Discussion Questions
Overall, has a lack of understanding of choice determination severely diminished survey findings? Can behavioral economics approaches be applied to the survey process? What do you think of nudging, heating or incentivizing more accurate survey responses?
Poll
Joel’s insights are great “food for thought.” As an industry, we don’t always understand the “before” side of the transaction as well as we need. It has felt to me like we are trying to solve for improving health through autopsies alone. We look at the purchase data and try to “back into” the other stuff (motivation, decision-making, jobs-to-be-done, etc.).
Behavioral economics and how people process information is of critical importance and often neglected. Surveys may be a wildly inaccurate approach to acquiring data given the social pressures, poor survey design and question construction among other things. Behavioral economics can provide some context—but the emphasis is on BEHAVIORAL, not cognitive alone.
It’s hard to argue with Joel’s premise that there is more room for behavioral study in marketing research. At the same time, it’s hard to paint “market research” with such a broad brush, since it’s driven by many different methodologies. Surely a lot of good market research is driven by behavioral observation in the field, just as other good research is drawn more from data about demographics, shopping history, etc.
It’s also important to keep some historical perspective about the risks of “research overreach.” Apple, for one, has been famously averse to market research before introducing products like the iPhone, iPad, and so forth. More behavioral research might have led a more cautious company to scrap these products before their launch.
Joel is spot on in pointing out the myriad of ways market research is influenced by the structure of the instrument—usually adversely. To Dick Seesel’s point, I think this is at least part of the reason companies like Apple and others eschew research—no data is better than misleading data.
Like all afflictions, recognizing the problem and naming it is the hardest first step. Let’s wish Joel and his contemporaries much success in helping the industry overcome these failings of our current methods.
Joel’s article is well done and right on point. By the way, the Foundations of Quality Initiative is continuing and I am privileged to be the Project Director as the findings are brought to light.
I would suggest that Joel’s assertions can be misinterpreted to an all or nothing conclusion. For example to acquire quality responses from respondents we need participants who are motivated by a social contract. However, incentives for survey taking are a cost of entry for virtually all respondents. They opt in for surveys if there is cash (or in kind) incentives but participate at the highest level if they are ALSO self motivated to do the right thing. I’ve just completed in person interviews with survey respondents after they’ve completed an online survey and they confirm this perspective. And to Joel’s point, poorly designed questionnaires can undermine even the most committed (to truth) respondent.
I’ve been writing questionnaires and designing research methodologies for many years (more than many years). It’s always been a desirable challenge for me to build in context that facilitates the ability of respondents to visualize the circumstances and provide the best and most accurate portrayal of their intended behavior or current opinion.
Consequently, going way back, it was always better for my research to show actual (rather than pictures of or no pictures of) shopping environments, packages, signage and all, to determine purchase choices. Behavioral economics goes a step further to build several alternative environments as well as cerebral options (such as the late payment concept at the nursery school). The findings can be profound. And I also recommend Dan Ariely’s book (he’s also a great speaker… I was pleased to have invited him to a conference several years ago) to all marketers and market researchers. He frames decision making in a unique and relevant way. However, not all research can be conducted in the same experimental manner. We must be judicious and examine our objectives, the monetary implications of the results and decisions dependent on the data to design an appropriate methodology that takes into account all of the above.
So Joel’s article is right on. Please read it, learn from it and use what is applicable for your own business needs. Not every conclusion needs to be applied to every research undertaking. Please realize that he is expanding for you the array of tools you have for quality and relevant marketing research.
Surveys asking consumers what they might do in certain situations are fraught with issues and have yielded to marketers and manufacturers inaccurate data upon which to stack important considerations. Finding ways to make surveys more accurate is laudable, so I am in favor of investigating techniques to improve the process.
I do find the concept of “heating up the respondent” to reveal what their choice may be under similar future conditions to be interesting. Of course, that may yield inaccurate results in future conditions as many people would not be selecting a product in a “heated state.”
Without behavioral research, any survey findings have limited use. Like the old adage says “as a drunken man uses lamp-posts…for support rather than illumination.”
Surveys must be carefully composed to minimize respondent decision biases. The severity of these many biases varies by culture and demographics. Responses to some questions need to be discounted because of these biases.
Researchers studying consumer behavior (including marketers) finally convinced economists to incorporate some behavior into their models. Think economists still have much to learn about behavior and marketers are way ahead of them.
Thanks for the wonderful adds via comments. One thing I would like to add is the work of Steven Hoeffler (Vanderbilt, UNC) on RNPs (really new products). His findings show the importance of “warming up the respondent” when testing a concept for a “really new product.”
All three points need/should be considered when building a survey. In particular, mood. This is why I always struggled with white room focus groups. They are not “in the moment” and they are frankly being nudged by just being part of the focus group.
This is why I am so excited about some of the new apps entering the market that ask survey questions to the right person at the right time.
The app knows the age, sex and that the shopper is in the grocery store. Maybe it even knows some past shopping behavior. As a marketer, I can get a more accurate answer from this shopper using a quick survey while she is in store.
There is a lot of potential in this space for the groups that get it right.
The understanding of choice determination has perplexed marketers for years. Behavioral economics techniques can, and have been used in consumer studies in the past. I think the article above may have some merit. IBM did some research around this and found some interesting insights. Though marketing is key to business success, understanding media effects on sales is extremely difficult. Our research in this area sought to understand how complex behaviors emerge in markets from the decisions made by a heterogeneous mix of consumers. Verification of brand choice behavior that is outside of the expected or current marketing theory prevailed.
We were able to do detailed analysis of complex macro-market phenomena such as price bubbles using simple trading models with agent-based simulations. Princeton University participated in our joint research using simulations and analysis to understand how various price signals affect stable markets.
We also did research on forecasting the ROI of various marketing campaigns and advertisements. This is a challenge because the responses from consumers to marketing investments are nonlinear. To simulate such a non-linear response, we model each consumer as an agent with a finite memory that selects actions using heuristics (Technologies employed by companies like DemandTec, etc.). These simulations can efficiently evaluate the ROI based on the cumulative results of the individual nonlinear responses.
As far as surveys mentioned in the article, I think some of the aspects that affect human behavior are coming closer to becoming well-understood.
At its heart, Behavioral Economics really is the study of something we are all familiar with: framing. We have all seen how the changing of a few words in a question can affect answers given during a survey or focus group. For example ask yourself the following two questions: Overall, how happy are you? Overall, how happy are you right now?
The first is a global assessment of your happiness; the second is an assessment of your current state. Learning about Behavioral Economics gives us a filter to identify these different frames and consciously use them to get the best answers in our research. Thank you for highlighting some of its findings, Joel!
No. The greatest minimization to survey findings is that our audiences are overwhelmed by so much information that it is difficult to accurately measure their backgrounds and minimize the impact of outside factors, including communications and influences.
More Discussions
Will Ryan Cohen Score GameStop’s $35 Billion Pay Package?
GameStop CEO Ryan Cohen has been offered a $35 billion pay package based around performance. Will…What Should Saks Global Try Next?
The trouble looks very uncertain for Saks. What is its next move?Did Holiday 2025 Mark a Shift Into the ‘Agentic Shopping’ Retail Era?
Salesforce data suggests that holiday 2025 was the retail period in which agentic shopping turned the…
Joel’s insights are great “food for thought.” As an industry, we don’t always understand the “before” side of the transaction as well as we need. It has felt to me like we are trying to solve for improving health through autopsies alone. We look at the purchase data and try to “back into” the other stuff (motivation, decision-making, jobs-to-be-done, etc.).
Behavioral economics and how people process information is of critical importance and often neglected. Surveys may be a wildly inaccurate approach to acquiring data given the social pressures, poor survey design and question construction among other things. Behavioral economics can provide some context—but the emphasis is on BEHAVIORAL, not cognitive alone.
It’s hard to argue with Joel’s premise that there is more room for behavioral study in marketing research. At the same time, it’s hard to paint “market research” with such a broad brush, since it’s driven by many different methodologies. Surely a lot of good market research is driven by behavioral observation in the field, just as other good research is drawn more from data about demographics, shopping history, etc.
It’s also important to keep some historical perspective about the risks of “research overreach.” Apple, for one, has been famously averse to market research before introducing products like the iPhone, iPad, and so forth. More behavioral research might have led a more cautious company to scrap these products before their launch.
Joel is spot on in pointing out the myriad of ways market research is influenced by the structure of the instrument—usually adversely. To Dick Seesel’s point, I think this is at least part of the reason companies like Apple and others eschew research—no data is better than misleading data.
Like all afflictions, recognizing the problem and naming it is the hardest first step. Let’s wish Joel and his contemporaries much success in helping the industry overcome these failings of our current methods.
Joel’s article is well done and right on point. By the way, the Foundations of Quality Initiative is continuing and I am privileged to be the Project Director as the findings are brought to light.
I would suggest that Joel’s assertions can be misinterpreted to an all or nothing conclusion. For example to acquire quality responses from respondents we need participants who are motivated by a social contract. However, incentives for survey taking are a cost of entry for virtually all respondents. They opt in for surveys if there is cash (or in kind) incentives but participate at the highest level if they are ALSO self motivated to do the right thing. I’ve just completed in person interviews with survey respondents after they’ve completed an online survey and they confirm this perspective. And to Joel’s point, poorly designed questionnaires can undermine even the most committed (to truth) respondent.
I’ve been writing questionnaires and designing research methodologies for many years (more than many years). It’s always been a desirable challenge for me to build in context that facilitates the ability of respondents to visualize the circumstances and provide the best and most accurate portrayal of their intended behavior or current opinion.
Consequently, going way back, it was always better for my research to show actual (rather than pictures of or no pictures of) shopping environments, packages, signage and all, to determine purchase choices. Behavioral economics goes a step further to build several alternative environments as well as cerebral options (such as the late payment concept at the nursery school). The findings can be profound. And I also recommend Dan Ariely’s book (he’s also a great speaker… I was pleased to have invited him to a conference several years ago) to all marketers and market researchers. He frames decision making in a unique and relevant way. However, not all research can be conducted in the same experimental manner. We must be judicious and examine our objectives, the monetary implications of the results and decisions dependent on the data to design an appropriate methodology that takes into account all of the above.
So Joel’s article is right on. Please read it, learn from it and use what is applicable for your own business needs. Not every conclusion needs to be applied to every research undertaking. Please realize that he is expanding for you the array of tools you have for quality and relevant marketing research.
Surveys asking consumers what they might do in certain situations are fraught with issues and have yielded to marketers and manufacturers inaccurate data upon which to stack important considerations. Finding ways to make surveys more accurate is laudable, so I am in favor of investigating techniques to improve the process.
I do find the concept of “heating up the respondent” to reveal what their choice may be under similar future conditions to be interesting. Of course, that may yield inaccurate results in future conditions as many people would not be selecting a product in a “heated state.”
Without behavioral research, any survey findings have limited use. Like the old adage says “as a drunken man uses lamp-posts…for support rather than illumination.”
Surveys must be carefully composed to minimize respondent decision biases. The severity of these many biases varies by culture and demographics. Responses to some questions need to be discounted because of these biases.
Researchers studying consumer behavior (including marketers) finally convinced economists to incorporate some behavior into their models. Think economists still have much to learn about behavior and marketers are way ahead of them.
Thanks for the wonderful adds via comments. One thing I would like to add is the work of Steven Hoeffler (Vanderbilt, UNC) on RNPs (really new products). His findings show the importance of “warming up the respondent” when testing a concept for a “really new product.”
All three points need/should be considered when building a survey. In particular, mood. This is why I always struggled with white room focus groups. They are not “in the moment” and they are frankly being nudged by just being part of the focus group.
This is why I am so excited about some of the new apps entering the market that ask survey questions to the right person at the right time.
The app knows the age, sex and that the shopper is in the grocery store. Maybe it even knows some past shopping behavior. As a marketer, I can get a more accurate answer from this shopper using a quick survey while she is in store.
There is a lot of potential in this space for the groups that get it right.
The understanding of choice determination has perplexed marketers for years. Behavioral economics techniques can, and have been used in consumer studies in the past. I think the article above may have some merit. IBM did some research around this and found some interesting insights. Though marketing is key to business success, understanding media effects on sales is extremely difficult. Our research in this area sought to understand how complex behaviors emerge in markets from the decisions made by a heterogeneous mix of consumers. Verification of brand choice behavior that is outside of the expected or current marketing theory prevailed.
We were able to do detailed analysis of complex macro-market phenomena such as price bubbles using simple trading models with agent-based simulations. Princeton University participated in our joint research using simulations and analysis to understand how various price signals affect stable markets.
We also did research on forecasting the ROI of various marketing campaigns and advertisements. This is a challenge because the responses from consumers to marketing investments are nonlinear. To simulate such a non-linear response, we model each consumer as an agent with a finite memory that selects actions using heuristics (Technologies employed by companies like DemandTec, etc.). These simulations can efficiently evaluate the ROI based on the cumulative results of the individual nonlinear responses.
As far as surveys mentioned in the article, I think some of the aspects that affect human behavior are coming closer to becoming well-understood.
At its heart, Behavioral Economics really is the study of something we are all familiar with: framing. We have all seen how the changing of a few words in a question can affect answers given during a survey or focus group. For example ask yourself the following two questions: Overall, how happy are you? Overall, how happy are you right now?
The first is a global assessment of your happiness; the second is an assessment of your current state. Learning about Behavioral Economics gives us a filter to identify these different frames and consciously use them to get the best answers in our research. Thank you for highlighting some of its findings, Joel!
No. The greatest minimization to survey findings is that our audiences are overwhelmed by so much information that it is difficult to accurately measure their backgrounds and minimize the impact of outside factors, including communications and influences.