Should Instacart discard shopper ratings from chronically grumpy customers?
Source: Instacart

Should Instacart discard shopper ratings from chronically grumpy customers?

Instacart has overhauled its customer ratings system to dilute the impact of hard-to-please customers and deliver a more “fair experience” to shoppers (independently-contracted delivery drivers).

As part of the update, Instacart will remove ratings if a customer consistently rates shoppers below five stars. Shoppers will also now only need to maintain a 4.7 average rating or above for batch prioritization in another move to reduce the impact of a single or a few low ratings.

Under the former system put in place just before the pandemic arrived, shoppers were prioritized batches based on whomever had the highest average rating. Instacart has said shoppers favored the system based on customer feedback over the prior system based on time spent shopping, or speed, because it put a priority on the quality of the customer interaction.

Shoppers have complained their ratings — and thereby access to the most profitable batches — were hurt by issues out of their control, including customers who tend to give low ratings as well as customers lying about having received their order. Some also claimed the low rating was tied to an out-of-stock situation, even though Instacart has said it removes such ratings from shopper scores.

An analysis from the Los Angeles Times from December 2020 found that shoppers whose ratings dropped slightly below 4.95 out of five stars saw significantly fewer and less lucrative batches.

Instacart’s system is based on the average customer star rating of the shopper’s last 100 orders, with the lowest rating removed.

Under the update, Instacart shoppers also gain new features, including a feedback loop to report issues with customers and now have an option to block certain customers from their list of batches moving forward.

Past articles have detailed how Uber and Lyft drivers with low scores risked receiving few rides or getting delisted from the app. Outside of the on-demand workplace, a recent NBC News article explored how Starbucks’ workers felt threatened with reduced store hours because of the cafe’s use of crowd-sourced ratings, called connection scores, to evaluate the customer service.

BrainTrust

"I don’t have a problem with Instacart adjusting ratings on things that are out of the shopper's control."

Georganne Bender

Principal, KIZER & BENDER Speaking


"Face it, we don’t live in a 5-star world."

Brian Cluster

Director of Industry Strategy - CPG & Retail, Stibo Systems


"I rate about 3/4 of shoppers with 5 stars, but here’s the thing: the bad shoppers on Instacart are really, really bad."

Jasmine Glasheen

Content Marketing Manager, Surefront


Discussion Questions

DISCUSSION QUESTIONS: Do think the adjustments to Instacart’s customer ratings are fairer to its shoppers as well as the end customer? Do you like the use of customer feedback ratings to gauge the performance of front-line employees, or are other methods more effective?

Poll

16 Comments
Oldest
Newest Most Voted
Inline Feedbacks
View all comments
DeAnn Campbell
Active Member
1 year ago

Building a scoring system around customer service excellence is a great way for Instacart to ensure they are building a system that has staying power. As with all Artificial Intelligence guided systems, it will take some time to train the algorithms — and train shoppers — to achieve the best results.

Shep Hyken
Active Member
1 year ago

All feedback, even from chronically unhappy customers, is valid to consider. You never know what that disgruntled curmudgeon (customer) might have to say that could be worth considering. That said, I like that the chronic low rater might be taken out of the mix when evaluating the Instacart shoppers. What I suggest to my clients when looking at ratings is to toss out the top and bottom 5 percent. Some people are always happy, and some people are never happy. However, the 90 percent in the middle will give you a good sense of how you’re doing. One suggestion in the rating system is to have an open-ended question, such as, “Why did you give us that rating?” A little qualitative feedback will give some relevance to the rating.

Georganne Bender
Noble Member
1 year ago

I use Instacart a few times a month to send groceries to an elderly relative in another state. I am grateful that Instacart shoppers can offer a service that I cannot and I rate them accordingly. But you only have to look at any online rating service to see how hyper critical people can be.

I don’t have a problem with Instacart adjusting ratings on things that are out of the shopper’s control. I also like that the shoppers have the ability to block problem customers. Customer service has changed since the pandemic and continues to evolve. We have to be flexible and fair to employees as well as customers.

Katie Thomas
1 year ago

This is one of the challenges of the gig economy – it’s a bit of a crap shoot (excuse my French) in terms of the quality consumers get. Some brands have experimented with pairing consumers with the same shoppers regularly to help shoppers better know consumer habits and how to shop for them (e.g. when they do and don’t want a replacement item). I’d challenge that holds more staying power than the traditional rating which, like always, can be heavily impacted by the extremes.

Ken Morris
Trusted Member
1 year ago

My experience with Instacart has not been the greatest, so I guess I’m one of the chronically grumpy customers. Maybe it was the time I ordered a package of 18 eggs and got none, because nobody thought of giving me a dozen and a half dozen instead. Seeing aisles clogged with Instacart shoppers instead of regular people during the pandemic annoyed me, too. But feedback from the end-customer is important. It’s supposed to be a valuable tool that lets businesses hear the voice of the customer so they can make continuous improvements (improving the business, not the customer!). In the olden days, that’s how businesses thrived.

Ratings and reviews of customers should not be driving which deals they get, or their level of service. Reviews should be used to hold service providers’ feet to the fire and actually deliver.

Doug Garnett
Active Member
1 year ago

The NPS based rating systems should be discarded altogether. But if they can’t do that yet, this is a good step forward.

But chronically grumpy? Someone who rates service only 4 out of 5? Ridiculous. That 4.7 rating minimum for batch is absurd.

DeAnn Campbell
Active Member
Reply to  Doug Garnett
1 year ago

I’ve never been a fan of the NPS rating system.

David Spear
Active Member
1 year ago

Like companies of the past, the new gig economy giants will figure this out. Let’s remember, Instacart was founded 10 years ago, and the pandemic drove its business into the stratosphere! I say keep all ratings – the good, the bad and the ugly. Pearls of wisdom can come from the unlikely.

George Anderson
Member
1 year ago

Based on grading it would appear as though shoppers should get five stars for just doing their job correctly. Anything below that signifies some level of problem with the order and, of course, there is nothing above five to give kudos for exceptional work.
Instacart needs to recalibrate its system and its customers understanding of it so that three stars are given for doing the job expected and those above and below reflect higher or lower levels of performance.

Jasmine Glasheen
Member
1 year ago

I was a regular Instacart shopper long before the pandemic. I rate about 3/4 of shoppers with 5 stars, but here’s the thing: the bad shoppers on Instacart are really, really bad. We’re talking decide-not-to-deliver-your-food-an-hour-after-it-was-supposed-to-arrive bad. The issue is so extreme, and the projected shipping windows so lengthy themselves, that I and many of my urban professional friends living in downtown San Diego have switched to GoPuff.

I’m not against putting chronic down-voters a little lower in the algorithms, but I wish Instacart had more follow up measures in place for the extreme outliers to ensure consistency. I’m interested in the measures the company is taking to protect time-strapped customers, as well as employees.

Ryan Mathews
Trusted Member
1 year ago

You don’t always get what you measure, and this is a perfect example. If you begin throwing out lower ratings — for reasons good and/or bad — you are guaranteeing that your service level ratings, etc., will show dramatic improvement. Whether or not that improvement reflects reality is purely coincidental at best. Maybe, for example, some shoppers get consistently poor ratings because — well — they are just terrible. Under this model those shoppers will start looking like stars. There are much, much better ways to do what Instacart is allegedly trying to accomplish.

Brian Cluster
Member
1 year ago

Face it, we don’t live in a 5-star world. Things happen such as product shortages, traffic, accidents, and things break. It would be best for Instacart to be as transparent as possible about these scores and aim for continuous improvement vs.gaming the better scores. I agree with George’s point about re-calibration and education to consumer about what constitutes a 5-star vs. a 3-star rating. Additionally, perhaps there is an opportunity to prune old scores that are no longer relevant to the current customer service conditions?

KhanhB
1 year ago

What ever you want to improve. Try it. I only get one batch offered per day after shopping over 5,000 orders to achieve recognition in Gold Tier anyway. Somehow I like to read feedback from my customers to feel good about myself. I don’t pay attention much about 5 stars. I also believed that customer wouldn’t rewrite compliments again every time. So if I shopped for recurring customers, he/she would assume they already gave me a good credit sometime before … no need to say it over.

Patricia Vekich Waldron
Active Member
1 year ago

When did 5 become default for OK? I think an overhaul of rating systems — especially for gig-economy workers — is in order. It will take time to recalibrate the metrics and algorithms, but having ratings go both ways, flagging chronic complainers, dropping “curve-breaker” scores and giving shoppers some latitude in who they serve are all good steps. It should be an iterative process and evolve over time.

Ricardo Belmar
Active Member
1 year ago

Seems the real issue is that their 5-star rating system isn’t the right rating system for this type of performance measurement. I understand the desire to make it simple for the customer, but this approach assumes that any shopper who does their job should get a 5-star rating by default and if they don’t it assumes there is something wrong with them. Is that the best we can do for a performance evaluation system? I would suggest Instacart come up with a new system instead of relying on a 5-star approach. Rating the performance of a shopper is not the same as rating the quality of broccoli — and that’s what they are trying to do.

Brad Halverson
Active Member
1 year ago

Instacart has an opportunity here to redesign their ratings system to work better, rather than take shortcuts or shut down customers.

Why not just have a simple YES/NO for the key issues? Was it delivered on time as promised? Was it left at/near my doorstep? Were all the items you ordered in the bag? Were they correct? Was your card charged properly?

You can still have the 1-5 star ratings for qualitative things like quality/freshness of food, cleanliness of packaging, interactions/friendliness of employees. If you see a pattern here, then yes, investigate.