Waymo tests AI driving system that learns from labeled data

Alphabet’s self-driving spinoff Waymo achieved some noteworthy milestones this year, in August surpassing 10 million real-world miles with its driverless cars and last week launching Waymo One, a commercial driverless taxi service. But its researchers have their eyes fixed on the future.

In a blog post published today on Medium, researchers Mayank Bansal and Abhijit Ogale detailed an approach to AI driver training that taps labeled data — that is to say, Waymo’s millions of annotated miles from expert driving demonstrations — in a supervised manner. (In this context, “supervised” refers to a machine learning technique in which both the input and desired output data are provided.)

“In recent years, the supervised training of deep neural networks using large amounts of labeled data has rapidly improved the state-of-the-art in many fields, particularly in the area of object perception and prediction, and these technologies are used extensively at Waymo,” the researchers wrote. “Following the success of neural networks for perception, we naturally asked ourselves the question: … can we train a skilled driver using a purely supervised deep learning approach?”

Above: Waymo’s AI system nudges around a parked car in a simulated environment.

Image Credit: Waymo

In an attempt to create a system capable of immitating an expert driver, they crafted a neural network — dubbed ChauffeurNet, appropriately — that learned to generate a driving trajectory by observing a combination of real and simulated data, including a map, surrounding objects, traffic lights states, and the past motions of cars. A low-level controller converted the ten-point trajectory to steering and acceleration commands, allowing the AI model to drive both real and digital cars.

The model was fed examples from “the equivalent of about 60 days of expert driving data,” using techniques that ensured it didn’t extrapolate from past motion and actually reacted to changes in the environment. In tests, it responded to traffic controls such as stop signs and traffic lights, but predictably performed poorly when exposed to situations it’d never seen before.

The problem, the researchers point out, is that driving demos obtained from real-world driving are biased — they contain only examples of driving in good situations. In order to teach the network to recover from edge cases, the team synthesized near-accidents and collisions with objects, the latter of which they paired with disincentives that encouraged the AI model to avoid them.

Above: Waymo’s model handling a turn.

Image Credit: Waymo

ChauffeurNet performed better in a simulated environment with the losses and synthesized examples taken into account, even managing to nudge around parked vehicles, stop for a traffic light transitioning from yellow to red, and recover from slight deviations in its trajectory. And when used to drive one of Waymo’s Chrysler Pacifica minivans on a real-world private test track, it successfully followed a curved lane and handled stop signs and turns.

“Fully autonomous driving systems need to be able to handle the long tail of situations that occur in the real world,” the researchers wrote. “The planner that runs on Waymo vehicles today uses a combination of machine learning and explicit reasoning to continuously evaluate a large number of possibilities and make the best driving decisions in a variety of different scenarios … Therefore, the bar for a completely machine-learned system to replace the Waymo planner is incredibly high, although components from such a system can be used within the Waymo planner, or can be used to create more realistic ‘smart agents’ during simulated testing of the planner.”