No one wants to go grocery shopping. It’s all heavy carts, complex layouts, and unfathomable product placement decisions – why is the sherry vinegar next to the olive bar?

The daunting, sisyphean task takes up enough time to make you resentful. Making you hate it less, and helping you find exactly what you’re looking for, is Instacart’s raison d’etre, says Jeremy Stanley, the company’s VP of data science, and chief wielder of the machine learned algorthims behind the scenes.

[Jeremy Stanley joins an A-list of speakers uncovering the disruptive power of AI at MobileBeat 2017, July 11-12. See the full line-up and grab last-minute tickets.]

Instacart represents an advancement in online grocery shopping service: It gives you millions of products from hundreds of retail partners, and then hooks you up with a personal shopper who makes that grocery list land on your doorstep.


AI Unleashed

An exclusive invite-only evening of insights and networking, designed for senior enterprise executives overseeing data stacks and strategies.


Learn More

Underneath the hood is a combination of machine-learned algorithms that monitor millions of grocery orders, looking at how consumers search, what they’re searching for, what decisions they make. The algorithms are then able to organize searches so that your preferred products are at the top of your searches, and surface the items you never knew you needed but now will probably die without. And they’re constantly testing and refining models for predicting products that a user will buy again, try for the first time or add to cart next during any single session.

One of the most interesting challenges in this, Stanley says, is uncovering intent.

“We’re trying to interpret what exactly a customer mean by ‘pickled organic baby carrots,'” he explains. “There are probably ten or 20 different directions you can go with that query, and it’s very contextually dependent and dependent upon the user.”

And that directly impacts the personal shopper’s experience and ability to select exactly what the shopper is looking for —  and Instacart’s ability to come up with a replacement the customer will actually be happy about, not vaguely disappointed in, or even pissed about.

The company’s personal shoppers are hitting 60+ retail partner store locations and tens of thousands of store locations, Stanley says. What’s the chance that, for instance, strawberries are going to be in stock and high-quality at any one of those store locations later this afternoon?

“That’s an unknown quantity,” he explains, “and we have to use machine learning to guide the shoppers to ensure the customer has the best experience possible.”

Maybe the customer doesn’t want any replacements, or maybe they’re okay not getting the organic or maybe they’re okay getting a different brand, or something that’s more or less expensive, he explains. There is a tremendous amount of data available to build these models, Stanley says (and they’re continuing to deeply invest in this particular area).

There’s the opt-in signal from the customer, where customers are given the opportunity to change their replacement preferences for specific items, once and done and remembered forever. You can indicate that if the shopper can’t finding this fat content of greek yogurt, they can choose another brand, or something that isn’t organic.

“The great thing about this is that it’s an opt-in signal from the customer — they’ve told us that’s a great replacement,” Stanley says. “The bad thing is that it requires the customer to do it, so a lot of times that’s just not going to happen.”

Allowing customers to make decisions in real-time had been considered, but that sets up expectations on both the customer’s side and the shopper’s side.

The challenge is two-fold, he says. First, it means that the customer has to be online and responsive. And if your whole system depends upon that, and you’re unable to reach the customer, it’s going to be perceived as a bad shopping experience because you don’t have the safeguards to deal with those cases.

The other problem: Shoppers are busy. Asking them to get each replacement decision authorized means exerting a lot of energy and slowing them down to a crawl in some cases, because communication is laggy and they’re forced to wait in the middle of the store hoping they can get in touch with their customer.

“A part of the role of machine learning is to simply automate things so that we don’t have to pester the customer and distract the shopper,” Stanley says.

To build the machine-learned algorithm that tackles this issue, they add data by capturing customer comments and responses to replacements. They’re also able to take the data streams that identify in real time if customers are more or less happy with, say, the strawberries they’re getting.

For example, if you’re finding that customers are complaining about the strawberries you’ve delivered from a specific store location over the last few hours, you can infer that perhaps you should stop picking them and force a replacement, because the quality’s gone down.

You can get a little bit more leading than that and get ahead of the issue by actually working with the retailer to see sales in the stores, Stanley says. If a high-moving item suddenly drops in sales, going from 15 per hour to three, and then down to one, that indicates that while there may still be some in stock, people have started buying them at a much less rapid rate, and probably the quality is not as good.

“With our retail partners, we provide a lot of data to help them understand what customers are searching for, to help them understand what we’re finding in their stores, and what we’re not finding in their stores so they can manage their stock better,” Stanley says. “There’s a lot of opportunities for collaboration with data between us as the digital provider and the physical grocery store to improve the operations, efficiency, and the selection of those grocery stores.”

Mistakes happen, Stanley says. “And the other part of machine learning is to make sure we don’t make the same mistakes over and over again.”

He points at the ways shoppers have become unexpectedly creative in the decisions around replacements.

“One of my favorites is shoppers replacing ghee with a pound of butter,” he says. “Ghee is clarified butter which takes hours to make. It’s kind of telling the customer here, go do it yourself. And surprise surprise, the majority of customers don’t really like that.”

They’ve seen butternut squash replacing spaghetti squash (good try), and shoppers selecting riced cauliflower from the frozen section instead of jasmine rice. There are endless examples — that get corrected quickly.

“It’s a really interesting problem where there are some things like that may be globally obvious, but in other cases they may be culturally specific or specific to the customer,” he says. “So the role of machine learning is pretty important to try and separate the signal from the noise and to generalize for customers and across different types of products.”