VIENA2: A Driving Anticipation Dataset

TL;DR

VIENA2 is a comprehensive large-scale dataset for driving action anticipation, featuring diverse scenarios, sensor data, and annotations, enabling improved research in automated driving safety and anticipation techniques.

Contribution

The paper introduces VIENA2, a new extensive dataset for driving action anticipation, and benchmarks state-of-the-art methods including a novel multi-modal LSTM architecture.

Findings

Dataset contains over 15K videos across 5 scenarios.

Includes 25 action classes with 600 samples each.

Benchmark results demonstrate effectiveness of proposed models.

Abstract

Action anticipation is critical in scenarios where one needs to react before the action is finalized. This is, for instance, the case in automated driving, where a car needs to, e.g., avoid hitting pedestrians and respect traffic lights. While solutions have been proposed to tackle subsets of the driving anticipation tasks, by making use of diverse, task-specific sensors, there is no single dataset or framework that addresses them all in a consistent manner. In this paper, we therefore introduce a new, large-scale dataset, called VIENA2, covering 5 generic driving scenarios, with a total of 25 distinct action classes. It contains more than 15K full HD, 5s long videos acquired in various driving conditions, weathers, daytimes and environments, complemented with a common and realistic set of sensor measurements. This amounts to more than 2.25M frames, each annotated with an action label, corresponding to 600 samples per action class. We discuss our data acquisition strategy and the statistics of our dataset, and benchmark state-of-the-art action anticipation techniques, including a new multi-modal LSTM architecture with an effective loss function for action anticipation in driving scenarios.