Reliving the Dataset: Combining the Visualization of Road Users' Interactions with Scenario Reconstruction in Virtual Reality

TL;DR

This paper introduces a methodology for analyzing traffic datasets by mapping scenes to semantic graphs for objective criticality detection and recreating scenarios in VR for subjective analysis, aiding automated vehicle development.

Contribution

It combines semantic scene graph mapping with VR scenario reconstruction to improve detection and understanding of critical traffic scenarios in datasets.

Findings

Semantic scene graphs enable automated critical scenario detection.

VR reconstruction allows detailed subjective analysis.

Method enhances dataset usability for automated vehicle safety.

Abstract

One core challenge in the development of automated vehicles is their capability to deal with a multitude of complex trafficscenarios with many, hard to predict traffic participants. As part of the iterative development process, it is necessary to detect criticalscenarios and generate knowledge from them to improve the highly automated driving (HAD) function. In order to tackle this challenge,numerous datasets have been released in the past years, which act as the basis for the development and testing of such algorithms.Nevertheless, the remaining challenges are to find relevant scenes, such as safety-critical corner cases, in these datasets and tounderstand them completely.Therefore, this paper presents a methodology to process and analyze naturalistic motion datasets in two ways: On the one hand, ourapproach maps scenes of the datasets to a generic semantic scene graph which allows for a high-level and objective analysis. Here,arbitrary criticality measures, e.g. TTC, RSS or SFF, can be set to automatically detect critical scenarios between traffic participants.On the other hand, the scenarios are recreated in a realistic virtual reality (VR) environment, which allows for a subjective close-upanalysis from multiple, interactive perspectives.