©Kaspars Grinvalds via Canva.com
January 5, 2024
Are Online or Phone Surveys Better?
A recent study from the Pew Research Center found errors in opt-in online surveys to be about twice that of probability-based panels due to “bogus respondents,” or those who usually say “Yes” regardless of the question.
Online, opt-in surveys have nonetheless been found to be increasingly used because they’re “typically fast, inexpensive, and convenient” in contrast to probability-based sampling methods, such as random-digit dialing of phones, according to research led by Stanford University.
The value of online versus phone surveys continues to be examined as internet polling has surpassed traditional phone-based methods.
Online surveys are generally seen as less expensive than phone surveys that require live interviewers and phone expenses. Online surveys can also be distributed more quickly to a wide range of potential respondents and compiled easily.
“The results of an online survey are always ready to be read and analyzed,” according to MarTech Series. “The best thing is you can do so in real-time, so the waiting period for the survey to give us the data analytics is gone.”
Beyond costs and limited reach, phone surveys can also be more time-consuming, and response rates have fallen over the years as caller ID has arrived.
Gallup analysis finds that with the interviewer able to engage respondents and control the survey’s pace, phone surveys tend to lead to “less forthcoming” responses to sensitive topics versus web surveys. The interaction also leads respondents to “give more extreme, positive responses to attitudinal items and more socially desirable responses.”
Still, studies have shown that telephone surveys are able to produce accurate results, with interviewers able to add clarification or explanation for questions that may be confusing to respondents, something that an online survey cannot do.
Drive Research suggests that phone surveys may be more effective in reaching older and “less tech-savvy” demographics, while online methods may work better to reach younger audiences. The objectives of the survey should also be considered. Drive Research stated, “If you need detailed responses and personalization, phone surveys may be the better option. If you need quick results and broad data, online surveys may be the better option.”
Discussion Questions
How would you rate the value of online versus phone survey methods for consumer research? What advice would you have for choosing which technique to use based on survey goals?
Poll
Loading …BrainTrust
Shep Hyken
Chief Amazement Officer, Shepard Presentations, LLC
Ryan Mathews
Founder, CEO, Black Monk Consulting
Brian Numainville
Principal, The Feedback Group
Recent Discussions
More Discussions
How Can Retailers Crack the Gen Z ‘Riddle’?
Gen Z is becoming an increasingly important consumer demographic, but an elusive one to map. How…Does Bath & Body Works’ Scented Holiday Campaign Pass the Sniff Test?
Bath & Body Works is kicking off a new campaign with scent being at the fore…Which Tech Will Solve In-Store Inventory Woes?
Three major tech lineups are looking to solve retail inventory problems. Which is most suited?
Every survey methodology has it advantages and disadvantages. Telephone surveys can be more accurate, but they are also more complex to administer and do not allow for responses to visual questions and so forth. Online surveys are more flexible and faster, but they need to have quality control checks built in – like checking for speeding, straight-lining responses, attention, etc – to be reliable.
All surveys contain some amount of bias. Telephone surveys, historically the staple of survey firms, have become challenged as land lines get cut in favor of mobile only generations. Spam and fraudsters have not helped the research industry as many consumers simply shun them all. Online surveys do offer automated tabulation and virtually immediate summary findings, not to mention a significantly lower cost. The best advice for choosing a survey methodology is that it depends on what insights you’re trying to acquire. If you want a broad survey and fast results online surveys are clearly the best choice; if you’re looking for in-depth insight into a specific market segment, focus groups or panels are generally better. Anyone who has ever managed a consumer research project well knows, almost all survey results need to carefully interpreted – findings are not absolutes, but rather directional.
Personally, I find online preferable, for the obvious reason that it’s easy(ier) to respond to an e-mail. Ultimately, tho, what matters is that you get the response you’re looking for, so if you’re trying to reach a demographic that’s still more tied to phones, then that may be what makes sense. Ultimately, as with any research, it’s important you have a means to validate your results; and while consumer research seldom hinges on the kind of precision that political polling does, a company that (simply) goes on faith with the results may be in for an unpleasant surprise.
Both methods are in trouble. I get survey requests ALL THE TIME and my stamina is gone. I have no interest in answering any longer and am willing to bet that I am not an outlier. I mean, every single interaction seem to merit a survey: “how was your shopping experience today?” “Hit a button to tell us how the bathroom was” “you recently called in an order, how was your experience?” and on and on and on it goes.
An as far as someone calling me. Never gets answered because of all of the other spam calls I get.
Stop already.
There is no national cellphone directory, which means that many phone surveys go only to landlines. How many millennials do you know who have landlines?
Only 27 percent of U.S. households have landlines. That leads to a significant sample bias.
Not true in terms of “many phone surveys go only to landlines.” There are many sources of cell phone sample and they provide accurate ways to reach people, within local markets, states, or nationally.
But anyone not on my personal cellphone directory goes immediately to voice mail. And from there, if I don’t know you, you’re blocked.
Amazing how we get hundreds of surveys per project done on cell phones then I guess. Kind of like a spam filter with online studies. There is always something to work around no matter the methodology..
I’ve found that the quick and dirty approach is fine if you don’t care about the accuracy of the results. How many of us have seen badly written questions, incorrect skip patterns, and overly long experiences that often miss the point you’d like to make to the questioner .Like today’s poll (no offense) to whoever wrote it – efficiency hasn’t been the question for years now – we all know that online is more efficient. The question is accuracy, and online CAN be notoriously inaccurate unless there’s a lot of data hygiene before anybody looks at the results.
Stephen,
Totally agree. Of course this begs the question, “If you don’t care about the accuracy of the result,” why bother with the survey? You’ve raised a key problem. Too many surveys are written by people who are only interested in confirmation of their self-interested hypothesis or by people who don’t know enough to create a good question. Garbage in, garbage out.
Online and phone surveys are both valuable methods for consumer research, but they have different strengths and weaknesses. Online surveys are fast, inexpensive, convenient, and can reach a wide and diverse audience. However, they may suffer from low quality responses, self-selection bias, and limited interaction. Phone surveys are more expensive, time-consuming, and limited in reach, but they can offer more clarity, personalization, and control over the survey process. They may also elicit more honest and nuanced responses from respondents. The choice of technique depends on the survey goals, budget, and target population.
Target demographic and survey objectives.should be considered when deciding which approach to use but when reporting it would be prudent to highlight the method used to avoid inappropriate comparisons.
The first bias one should deal with with either type of survey is the sample bias. People who answer surveys are different from people who won’t. (Read Mark Self’s comments above.)
Assuming one understands the biases built into the questions, the phone works better for long-form questions. Short questions are somewhat appealing for “check the box.”
If you are the respondent, online surveys are preferable and and if you are the marketer more efficient. Telephone surveys, conversely are less preferable to the consumer (consumer does not control the situation) but more effective for the marketer (all the questions are answered and it is probably harder to abandon before completion.
Every survey instrument and methodology has its pros and cons. Online is fast and can be done quickly. The cost of calling customers is far higher than an online survey. Online can scale much easier (for numbers and speed) compared to telephone. Thanks to technology, quality issues are not an issue compared to the recent past. I still believe a fast/short online survey of a large number of customers yields a good and accurate result. An important point to remember is that data is the beginning of a future action. If all you do is collect data and don’t take action, you’re wasting everyone’s time.
Since we’ve been running online surveys for 20 years, I have a certain vested interest in saying they’re accurate, but they seem to pass the sniff test. We know when we have bad data we also know when it rings true
i will say this: survey design is an art form. It helps to have a methodology to create them (in our case, it’s the BOOT) and avoid answers with too many options or questions with 30 different lines to be checked off. Too many companies are “rolling their own” these may seem cheaper, but often they’re subtly geared to get the answer you want. But in any case, there are people who do online surveys for a living. It’s a thing.
PS. I don’t find telephone surveys useful for anyone
Like many BrainTrusters I’ve done lots and lots of surveys over the years. I have almost no confidence in most internet intercept survey results particularly when the sample sizes are 1,000 or less. The Digital Age birthed the Professional Respondent. We have all become used to participating in online surveys to get something, and I suspect many of us have … well … fibbed or rushed a bit. Phone surveys can be better, depending on who does them, but they have their issues as well. The two most reliable methods I’ve used involve psychometrics — extremely accurate but scarily expensive and ethnography conducted by real anthropologists, not marketers who watched a YouTube video. To me the bottom line is the size and integrity of the sample (hence the psychometricians) and the talent, skills, and techniques of the interviewer/observer (hence the ethnographer). After that the data usually doesn’t pass my “sniff test.”
Yes, that’s why survey design is an art form. I can remember doing designs on the screens when there were too many questions. I will say 1,000 respondents aren’t necessary. I find in aggregate, when talking to retailers, the numbers start to normalize in aggravate at about 40 respondents. By 80, you can do some decent data cuts.
Paula,
Totally agree about the criticality of survey design. That’s why survey design needs to be left to the experts. As to sample size I think that depends on two variables — the size of the issue you are addressing and how projectable you need the results to be. When I see these “surveys” of 1,000 with maybe 730 actual respondents proclaiming the “truth” or some new insight on a complex national or global issue my flesh crawls. That said, a very well done survey on a very focused topic doesn’t require millions of respondents, again pointing out the need to have people running the survey that know what they are doing.
As someone who has been doing research using every method under the sun for the last 30+ years, there is a time and a place for all methods. Online is great for some contexts while phone still works well in others. And of course this is a simplification of the many nuanced methods that are available within both of these categories. What is important is to carefully evaluate what you are trying to learn, who you need to learn that from, and the best vehicle to reach them. Plus, DIY tools are great…in the hands of someone who knows how to use them. Churning out a survey when you don’t have the background to do so equals garbage in, garbage out.
Online survey is like The Flash—super fast! Phone survey is likewise wizards, it takes its time and get into the nitty-gritty details. So, it’s like a battle between speed and wisdom in understanding people.
Match your survey goals with the right tool, for insightful results. Need quick, broad insights. Go online, it’s the speedy racer. Want detailed, personal stuff. Choose phone, it’s like a friendly chat.
Sure, online surveys have heir flaws. But I think people would be more likely to take an online survey than a phone survey, especially when the call comes at dinner time, at 8 am, or at 8 pm.
John, I agree, but that’s a problem by itself. The people who routinely take online surveys are about as reliable as folks that show up for every focus group they can squeeze into. Easier doesn’t necessarily equal better or even as good. That’s why, when possible, I’d rather stick to ethnography.
My experience has been a greatly reduced margin of error with online surveys mostly because participants enter their responses directly into the system. Other methods are reliant on the attentiveness of staff to enter all details correctly.
Online surveys are less expensive, faster, more accurate, quicker to analyze, user-friendly, more honest, selective, and more flexible.
Respondents sometimes answer the same question differently depending on the format of the interview. Db
Deciding between online and phone surveys depends on several factors, and each method has its advantages and disadvantages. For example, online surveys can be more cost-effective than traditional phone surveys because there’s no interviewer, but online surveys also have accessibility limitations based on WiFi connectivity. Phone surveys offer more opportunities to explain the context, fostering a deeper understanding and reaching a broader audience. That is, of course, dependent on whether the intended survey respondent answers the phone and stays on the line. My advice for choosing which route to take is to consider your target population, budget constraints, the nature of the research questions, and the desired level of respondent engagement. In some cases, a combination of both methods (mixed-mode surveys) may be used to mitigate the limitations of each method and maximize the benefits of both.