ALOOD: Exploiting Language Representations for LiDAR-based Out-of-Distribution Object Detection

TL;DR

This paper introduces ALOOD, a novel method that leverages language representations from vision-language models to improve out-of-distribution object detection in LiDAR data for autonomous driving, treating it as a zero-shot classification problem.

Contribution

It proposes a new approach that aligns LiDAR object features with language model features, enabling effective OOD detection without retraining on OOD data.

Findings

Achieves competitive results on the nuScenes OOD benchmark.

Introduces a zero-shot OOD detection framework for LiDAR data.

Demonstrates the effectiveness of language representations in OOD detection.

Abstract

LiDAR-based 3D object detection plays a critical role for reliable and safe autonomous driving systems. However, existing detectors often produce overly confident predictions for objects not belonging to known categories, posing significant safety risks. This is caused by so-called out-of-distribution (OOD) objects, which were not part of the training data, resulting in incorrect predictions. To address this challenge, we propose ALOOD (Aligned LiDAR representations for Out-Of-Distribution Detection), a novel approach that incorporates language representations from a vision-language model (VLM). By aligning the object features from the object detector to the feature space of the VLM, we can treat the detection of OOD objects as a zero-shot classification task. We demonstrate competitive performance on the nuScenes OOD benchmark, establishing a novel approach to OOD object detection in LiDAR using language representations. The source code is available at https://github.com/uulm-mrm/mmood3d.