Third Wave: Perception AI

Before artificial intelligence, all machines were deaf and blind. Sure, you could take digital photos or make audio recordings, but these merely reproduced our audio and visual environments for humans to interpret—the machines themselves couldn’t make sense of these reproductions. To a normal computer, a photograph is just a meaningless splattering of pixels they must store. To an iPhone, a song is just a series of zeros and ones that it must play for a human to enjoy.

This all changed with the advent of perception AI.

Algorithms can now group the pixels from a photo or video into meaningful clusters and recognize objects in much the same way our brain does: golden retriever, traffic light, your brother Patrick, and so on. The same goes for audio data. Instead of merely storing audio files as collections of digital bits, algorithms can now both pick out words and often parse the meaning of full sentences.

Third-wave AI is all about extending and expanding this power throughout our lived environment, digitizing the world around us through the proliferation of sensors and smart devices. These devices are turning our physical world into digital data that can then be analyzed and optimized by deep-learning algorithms. Amazon Echo is digitizing the audio environment of peoples’ homes. Alibaba’s City Brain is digitizing urban traffic flows through cameras and object- recognition AI. Apple’s iPhone X and Face++ cameras perform that same digitization for faces, using the perception data to safeguard your phone or digital wallet.

Blurred Lines and Our “OMO” World

As a result, perception AI is beginning to blur the lines separating the online and offline world. It does that by dramatically expanding the nodes through which we interact with the internet. Before perception AI, our interactions with the online world had to squeeze through two very narrow chokepoints: the keyboards on our computers or the screen on our smartphones. Those devices act as portals to the vast knowledge stored on the world wide web, but they are a very clunky way to input or retrieve information, especially when you’re out shopping or driving in the real world.

As perception AI gets better at recognizing our faces, understanding our voices, and seeing the world around us, it will add millions of seamless points of contact between the online and offline worlds. Those nodes will be so pervasive that it no longer makes sense to think of oneself as “going online.” When you order a full meal just by speaking a sentence from your couch, are you online or offline? When your refrigerator at home tells your shopping cart at the store that you’re out of milk, are you moving through a physical world or a digital one?

I call these new blended environments OMO: online-merge-offline. OMO is the next step in an evolution that already took us from pure e-commerce deliveries to O2O (online-to-offline) services. Each of those steps has built new bridges between the online world and our physical one, but OMO constitutes the full integration of the two. It brings the convenience of the online world offline and the rich sensory reality of the offline world online. Over the coming years, perception AI will turn shopping malls, grocery stores, city streets, and our homes into OMO environments. In the process, it will produce some of the first applications of artificial intelligence that will feel truly futuristic to the average user.

Some of these are already here. One KFC restaurant in China recently teamed up with Alipay to pioneer a “pay-with-your-face” option at some stores. Customers place their own order at a digital terminal, and a quick facial scan connects their order to their Alipay account—no cash, cards, or cell phones required. The AI powering the machines even runs a quick “liveness algorithm” to ensure no one can use a photograph of someone else’s face to pay for a meal. Pay-with-your-face applications are fun, but they are just the tip of the OMO iceberg.

Posted by Dr. Kai-Fu Lee on Feb 07, 2019 in All Posts Interviews with Dr. Lee

AI is Going to Change Everything

Dr. Lee Shares the 4 Coming Advances in AI

Download the Free White Paper