MAN TruckScenes: A multimodal dataset for autonomous trucking in diverse conditions

TL;DR

MAN TruckScenes is the first comprehensive multimodal dataset tailored for autonomous trucking, featuring diverse environmental conditions, detailed annotations, and novel sensor data including 4D radar, to advance research in truck-specific perception challenges.

Contribution

This work introduces MAN TruckScenes, the first large-scale multimodal dataset specifically designed for autonomous trucks, including unique sensor data and detailed annotations for truck-related perception tasks.

Findings

High-quality 3D bounding box annotations for 27 classes

First dataset to include 4D radar data with 360° coverage

Extensive analysis and baseline results provided

Abstract

Autonomous trucking is a promising technology that can greatly impact modern logistics and the environment. Ensuring its safety on public roads is one of the main duties that requires an accurate perception of the environment. To achieve this, machine learning methods rely on large datasets, but to this day, no such datasets are available for autonomous trucks. In this work, we present MAN TruckScenes, the first multimodal dataset for autonomous trucking. MAN TruckScenes allows the research community to come into contact with truck-specific challenges, such as trailer occlusions, novel sensor perspectives, and terminal environments for the first time. It comprises more than 740 scenes of 20s each within a multitude of different environmental conditions. The sensor set includes 4 cameras, 6 lidar, 6 radar sensors, 2 IMUs, and a high-precision GNSS. The dataset's 3D bounding boxes were manually annotated and carefully reviewed to achieve a high quality standard. Bounding boxes are available for 27 object classes, 15 attributes, and a range of more than 230m. The scenes are tagged according to 34 distinct scene tags, and all objects are tracked throughout the scene to promote a wide range of applications. Additionally, MAN TruckScenes is the first dataset to provide 4D radar data with 360{\deg} coverage and is thereby the largest radar dataset with annotated 3D bounding boxes. Finally, we provide extensive dataset analysis and baseline results. The dataset, development kit, and more are available online.