'Where Every Shopping Cart Knows Your Name'

To get a sense of where AI is headed, let’s take a quick trip just a few years into the future to see what a supermarket fully outfitted with perception AI devices might look like.

“Nihao, Kai-Fu! Welcome back to Yonghui Superstore!”

It’s always a nice feeling when your shopping cart greets you like an old friend. As I pull the cart back from the rack, visual sensors embedded in the handlebar have already completed a scan of my face and matched it to a rich, AI-driven profile of my habits, as a foodie, a shopper, and a husband to a fantastic cook of Chinese food. While I’m racking my brain for what groceries we’ll need this week, a screen on the handlebar lights up.

“On the screen is a list of your typical weekly grocery purchase,” the cart announces. And like that, our family’s staple list of groceries appears on the screen: fresh eggplant, Sichuan pepper, Greek yogurt, skim milk, and so on.

My refrigerator and cabinets have already detected what items we’re short on this week, and they automatically ordered the nonperishable staples—rice, soy sauce, cooking oil—for bulk delivery. That means grocery stores like Yonghui can tailor their selection around the items you’d want to pick out for yourself: fresh produce, unique wines, live seafood. It also allows the supermarkets to dramatically shrink their store’s footprint and place smaller stores within walking distance of most homes.

“Let me know if there’s anything you’d like to add or subtract from the list,” the cart chimes in. “Based on what’s in your cart and your fridge at home, it looks like your diet will be short on fiber this week. Shall I add a bag of almonds or ingredients for a split-pea soup to correct that?”

“No split pea soup but have a large bag of almonds delivered to my house, thanks.” I’m not sure an algorithm requires thanking, but I do it out of habit. Scanning the list, I make a couple of tweaks. My daughters are out of town so I can cut a few items, and I’ve already got some beef in my fridge so I decide to make my mother’s recipe of beef noodles for my wife.

“Subtract the Greek yogurt and switch to whole milk from now on. Also, add the ingredients for beef noodles that I don’t already have at home.”

“No problem,” it replies while adjusting my shopping list. The cart is speaking Mandarin Chinese, but in the perfectly synthesized voice of my favorite actress, Jennifer Lawrence. It’s a nice touch, and one of the reasons running errands doesn’t feel like such a chore anymore.

The cart moves autonomously through the store, staying a few steps ahead of me while I pick out the ripest eggplants and the most fragrant Sichuan peppercorns, key to creating the numbing spice in the beef noodles. The cart then leads me to the back of the store where a precision- guided robot kneads and pulls fresh noodles for me. As I place them in the cart, depth-sensing cameras on the cart’s rim recognize each item, and sensors lining the bottom weigh them as they go in.

The screen crosses things off as I go and displays the total cost. The precise location and presentation of every item has been optimized based on perception and purchase data gathered at the store: What displays do shoppers walk right by? Where do they stop and pick up items to inspect? And which of those do they finally purchase? That matrix of visual and business data gives AI-enabled supermarkets the same kind of rich understanding of consumer behavior that was previously reserved for online retailers.

Rounding the corner toward the wine aisle, a friendly young man in a concierge uniform approaches.

“Hi, Mr. Lee, how’ve you been?” he says. “We’ve just got in a shipment of some fantastic Napa wines. I understand that your wife’s birthday is coming up, and we wanted to offer you a 10 percent discount on your first purchase of the 2014 Opus One. Your wife normally goes for Overture, and this is the premium offering from that same winery. It has some wonderful flavors, hints of coffee and even dark chocolate. Would you like a tasting?”

He knows my weakness for California wines, and I take him up on the offer. It’s indeed fantastic.

“I love it,” I say, handing the wine glass to the young man. “I’ll take two bottles.”

“Excellent choice—you can continue with your shopping, and I’ll bring those bottles to you in just a moment. If you’d like to schedule regular deliveries to your home or need recommendations on what else to try, you can find those in the Yonghui app or with me here.”

All the concierges are knowledgeable, friendly, and trained in the art of the upsell. It’s far more socially engaged work than traditional supermarket jobs, with all employees ready to discuss recipes, farm-to-table sourcing, and how each product compares with what I’ve tried in the past.

The shopping trip goes on like this, with my cart leading me through our typical purchases, and concierges occasionally nudging me to splurge on items that algorithms predict I’ll like. As a concierge is bagging my goods, my phone buzzes with this trip’s receipt in my WeChat Wallet. When they’re finished, the shopping cart guides itself back to its rack, and I stroll the two blocks home to my family.

Perception AI-powered shopping trips like this will capture one of the fundamental contradictions of the AI age before us: it will feel both completely ordinary and totally revolutionary. Much of our daily activity will still follow our everyday established patterns, but the digitization of the world will eliminate common points of friction and tailor services to each individual. They will bring the convenience and abundance of the online world into our offline reality. Just as important, by understanding and predicting the habits of each shopper, these stores will make major improvements in their supply chains, reducing food waste and increasing profitability.

And a supermarket like the one described above isn’t far off. The core technologies already exist, and it’s largely a matter now of working out minor kinks in the software, integrating the back end of the supply chain, and building out the stores themselves.

Posted by Dr. Kai-Fu Lee on Feb 12, 2019 in All Posts AI and You

The Real AI Crisis

Dr. Lee Gives His Expert Analysis of How AI Will Impact the World

Download the Free White Paper