ROAD-Waymo: Action Awareness at Scale for Autonomous Driving

TL;DR

ROAD-Waymo is a large, comprehensive dataset designed to advance understanding of actions and interactions in autonomous driving scenes, enabling better perception and decision-making for AVs.

Contribution

It introduces the extensive ROAD-Waymo dataset with novel annotation pipeline and supports domain adaptation through the ROAD++ benchmark.

Findings

Dataset contains 198k frames, 54k agent tubes, 3.9M bounding boxes.

Enhanced annotation quality via a novel automatic violation detection pipeline.

Supports cross-country domain adaptation with the ROAD++ benchmark.

Abstract

Autonomous Vehicle (AV) perception systems require more than simply seeing, via e.g., object detection or scene segmentation. They need a holistic understanding of what is happening within the scene for safe interaction with other road users. Few datasets exist for the purpose of developing and training algorithms to comprehend the actions of other road users. This paper presents ROAD-Waymo, an extensive dataset for the development and benchmarking of techniques for agent, action, location and event detection in road scenes, provided as a layer upon the (US) Waymo Open dataset. Considerably larger and more challenging than any existing dataset (and encompassing multiple cities), it comes with 198k annotated video frames, 54k agent tubes, 3.9M bounding boxes and a total of 12.4M labels. The integrity of the dataset has been confirmed and enhanced via a novel annotation pipeline designed for automatically identifying violations of requirements specifically designed for this dataset. As ROAD-Waymo is compatible with the original (UK) ROAD dataset, it provides the opportunity to tackle domain adaptation between real-world road scenarios in different countries within a novel benchmark: ROAD++.