Scenario Extraction from a Large Real-World Dataset for the Assessment of Automated Vehicles

TL;DR

This paper presents a method for extracting relevant traffic scenarios from large real-world datasets to improve the safety assessment of automated vehicles, using data preprocessing, tagging, and scenario search techniques.

Contribution

It introduces a novel approach combining data reconstruction, activity tagging, and scenario extraction, applied to the Waymo dataset, to facilitate scenario-based AV testing.

Findings

Successful extraction of scenarios from large datasets

Application to Waymo Open Motion Dataset demonstrates feasibility

Open-source code and scenarios support further research

Abstract

Many players in the automotive field support scenario-based assessment of automated vehicles (AVs), where individual traffic situations can be tested and, thus, facilitate concluding on the performance of AVs in different situations. Since a large number of different scenarios can occur in real-world traffic, the question is how to find a finite set of relevant scenarios. Scenarios extracted from large real-world datasets represent real-world traffic since real driving data is used. Extracting scenarios, however, is challenging because (1) the scenarios to be tested should assess the AVs behave safely, which conflicts with the fact that the majority of the data contains scenarios that are not interesting from a safety perspective, and (2) extensive data processing is required, which hinders the utilization of large real-world datasets. In this work, we propose an approach for extracting scenarios from real-world driving data. The first step is data preprocessing to tackle the errors and noise in real-world data by reconstructing the data. The second step performs data tagging to label actors' activities, their interactions with each other and the environment. Finally, the scenarios are extracted by searching for combinations of tags. The proposed approach is evaluated using data simulated with CARLA and applied to a part of a large real-world driving dataset, i.e., the Waymo Open Motion Dataset (WOMD). The code and scenarios extracted from WOMD are open to the research community to facilitate the assessment of the automated driving functions in different scenarios.