A study by Interaction Metrics has concluded that most major retailers are horrible at conducting surveys.
The study examined customer satisfaction surveys of 51 top U.S. retailers (excluding grocers and warehouse clubs). Based on an objective evaluation of 15 elements, the average survey scored a 43 out of 100 points, an F grade.
Retailers were found to be worst at "Information Accuracy," with a 27 percent score on average. Measuring the neutrality or bias of data, Information Accuracy was worth 50 percent of the overall survey score. Infractions in this category included leading questions, biased language, double-barreled questions, title neutrality, question relevance and faulty scales.
According to the study, about a third of all questions led customers to give answers that companies wanted to hear. The authors presented an example of a leading question from Ace Hardware: “How satisfied were you with the speed of our checkout.” An example of forced wording from Gap included asking customers whether they agree with the statement, “The look and feel of the store environment was very appealing.”
In an example of faulty scale from Dollar General, the chain asked about customer service levels and provided five answer choices: extremely satisfied, very satisfied, somewhat satisfied, somewhat unsatisfied, very unsatisfied. The midpoint winds up positively skewed.
Retailers scored a 57 in "Customer Engagement, which was worth 35 percent of the survey score. This category measured such things as thoughtful welcomes in surveys, the use of jargon, survey length, progress transparency and customization. With 23 questions on average, the surveys were found to be excessively long. A Nordstrom survey advertised it would take two minutes to complete but the 25 questions took four to five minutes.
Some smaller aspects being judged included:
- "Branding Cues" (worth 10 percent of the score) graded style, spelling and grammar. The average score was 67.
- "Ease of Access" (worth 5 percent) graded how easy it is to locate and take the survey. The average score was 69. Examples of infractions included asking introductory questions irrelevant to the customer experience and requiring receipt codes.