Multimodal Detection of Unknown Objects on Roads for Autonomous Driving

TL;DR

This paper presents a multimodal approach combining lidar and camera data to detect unknown objects on roads, addressing a critical challenge for autonomous vehicles operating in unpredictable environments.

Contribution

It introduces a novel pipeline that integrates state-of-the-art detection models across sensor modalities for improved unknown object detection in autonomous driving.

Findings

Effective detection of unknown objects demonstrated on Waymo dataset

Identifies current gaps in anomaly detection research

Highlights the benefits of multimodal sensor fusion

Abstract

Tremendous progress in deep learning over the last years has led towards a future with autonomous vehicles on our roads. Nevertheless, the performance of their perception systems is strongly dependent on the quality of the utilized training data. As these usually only cover a fraction of all object classes an autonomous driving system will face, such systems struggle with handling the unexpected. In order to safely operate on public roads, the identification of objects from unknown classes remains a crucial task. In this paper, we propose a novel pipeline to detect unknown objects. Instead of focusing on a single sensor modality, we make use of lidar and camera data by combining state-of-the art detection models in a sequential manner. We evaluate our approach on the Waymo Open Perception Dataset and point out current research gaps in anomaly detection.