Spotting the Unexpected (STU): A 3D LiDAR Dataset for Anomaly Segmentation in Autonomous Driving

TL;DR

This paper introduces Spotting the Unexpected (STU), a pioneering 3D LiDAR dataset with dense semantic labels for anomaly segmentation in autonomous driving, addressing a critical gap in 3D anomaly detection research.

Contribution

It provides the first publicly available 3D anomaly segmentation dataset with multimodal data and sequential information, enabling improved safety in autonomous vehicle navigation.

Findings

Baseline models reveal challenges in 3D anomaly detection.

Dataset facilitates benchmarking and development of new methods.

Open access promotes further research in AV safety.

Abstract

To operate safely, autonomous vehicles (AVs) need to detect and handle unexpected objects or anomalies on the road. While significant research exists for anomaly detection and segmentation in 2D, research progress in 3D is underexplored. Existing datasets lack high-quality multimodal data that are typically found in AVs. This paper presents a novel dataset for anomaly segmentation in driving scenarios. To the best of our knowledge, it is the first publicly available dataset focused on road anomaly segmentation with dense 3D semantic labeling, incorporating both LiDAR and camera data, as well as sequential information to enable anomaly detection across various ranges. This capability is critical for the safe navigation of autonomous vehicles. We adapted and evaluated several baseline models for 3D segmentation, highlighting the challenges of 3D anomaly detection in driving environments. Our dataset and evaluation code will be openly available, facilitating the testing and performance comparison of different approaches.