Waymo expands autonomous driving data set and launches $110,000 challenge

In August 2019, Wamyo released the Waymo Open Dataset, which Waymo principal scientist Dragomir Anguelov claimed at the time was the largest multimodal corpus for autonomous driving.  The Alphabet subsidiary since augmented the set with an additional 10.6 million 2D labels (up from 1.2 million labels). Today, Waymo made available an expanded version of the corpus with an additional 800 segments, which Waymo expects will make it easier for researchers to build safe self-driving models from scratch without incurring the cost of collecting their own data.

Waymo also announced that as a part of the new Waymo Open Dataset competition, which launches today, it will award up to $110,000 in cash prizes across five object detection and tracking challenges. Winners will have the opportunity to present their work at Waymo’s Workshop on Scalability in Autonomous Driving at the CVPR 2020 conference in Seattle, which is scheduled to take place on June 14. (Waymo says another “suitable venue” might be chosen, depending on COVID-19-related developments.)

“Even as COVID-19 continues to develop, we are committed to fostering an environment of innovation and learning,” wrote Anguelov in a blog post, noting that the data set has been augmented with an additional 10 million-plus camera labels and the source code for Waymo’s camera model with rolling shutter compensation. He said the company hopes that this environment “can continue to grow and thrive in our temporarily virtual world.”

The Waymo Open Dataset contains data collected over the millions of miles Waymo’s cars have driven in Phoenix, Arizona; Kirkland, Washington; and Mountain View and San Francisco in California, and it covers a wide variety of urban and suburban environments during day and night, dawn and dusk, and sunshine and rain. Samples are divided into 1,000 driving segments, each of which captures 20 seconds of continuous driving corresponding to 200,000 frames at 10Hz through the sensors affixed to every Waymo car. These include five custom-designed lidars (which bounce light off objects to map them three-dimensionally) and five front- and side-facing cameras.

The corpus also has labeled lidar frames and images with vehicles, pedestrians, cyclists, and signage. The camera and lidar frames have been synchronized by in-house 3D perception models that fuse data from multiple sources, Waymo says, eliminating the need for manual alignment.

The Waymo Open Dataset challenges are as follows:

• 2D Detection: Given a set of camera images, produce a set of 2D boxes for the objects in the scene.
• 2D Tracking: Given a temporal sequence of camera images, produce a set of 2D boxes, their velocities (optional), and the correspondence between boxes across frames.
• 3D Detection: Given one or more lidar range images and the associated camera images, produce a set of 3D upright boxes for the objects in the scene.
• 3D Tracking: Given a temporal sequence of lidar and camera data, produce a set of 3D upright boxes, their velocities (optional), and the correspondence between boxes across frames.
• Domain Adaptation: Similar to the 3D Detection Challenge, but we provide additional segments from a new location and only a subset have labels.

Waymo will offer cash prizes to the top submissions in each of the five challenges as follows:

• 1st Place: $15,000
• 2nd Place: $5,000
• 3rd: Place: $2,000

The competition will run until May 31, 2020, and the leaderboards will remain active for future submissions. Waymo says all submissions will be evaluated through an automated process and that participants will receive their score in a matter of minutes.

According to the rules, participants who wish to be eligible for the prizes grant a license to Waymo to use their intellectual property. It’s standard language for this sort of contest, but it’s perhaps also a reflection of the cutthroat nature of the autonomous vehicle industry. Waymo recently raised $2.25 billion to scale up its operations, but the company’s annual cost is estimated at around $1 billion, while its robo-taxi business in Phoenix, Arizona — Waymo One — reportedly yields just hundreds of thousands of dollars a year in revenue.

The update to Waymo’s data set comes after Lyft open-sourced its own corpus for autonomous vehicle development. Other such data sets include nuScenes, Mapillary Vistas’ corpus of street-level imagery, the Canadian Adverse Driving Conditions (CADC), the KITTI collection for mobile robotics and autonomous driving research, and the Cityscapes data set developed and maintained by Daimler, the Max Planck Institute for Informatics, and the TU Darmstadt Visual Inference Group.