Out-of-Distribution Semantic Occupancy Prediction

TL;DR

This paper introduces a novel framework for out-of-distribution semantic occupancy prediction in 3D environments, enhancing anomaly detection in autonomous driving with synthetic datasets and a refined prediction model.

Contribution

It proposes a new OoD detection framework, OccOoD, and synthetic anomaly augmentation datasets, VAA-KITTI and VAA-KITTI-360, for improved safety in autonomous driving.

Findings

Achieves state-of-the-art OoD detection with AuROC of 65.50%

Maintains competitive semantic prediction performance

Demonstrates strong generalization in urban scenes

Abstract

3D semantic occupancy prediction is crucial for autonomous driving, providing a dense, semantically rich environmental representation. However, existing methods focus on in-distribution scenes, making them susceptible to Out-of-Distribution (OoD) objects and long-tail distributions, which increases the risk of undetected anomalies and misinterpretations, posing safety hazards. To address these challenges, we introduce Out-of-Distribution Semantic Occupancy Prediction, targeting OoD detection in 3D voxel space. To fill dataset gaps, we propose a Realistic Anomaly Augmentation that injects synthetic anomalies while preserving realistic spatial and occlusion patterns, enabling the creation of two datasets: VAA-KITTI and VAA-KITTI-360. Then, a novel framework that integrates OoD detection into 3D semantic occupancy prediction, OccOoD, is proposed, which uses Cross-Space Semantic Refinement (CSSR) to refine semantic predictions from complementary voxel and BEV representations, improving OoD detection. Experimental results demonstrate that OccOoD achieves state-of-the-art OoD detection with an AuROC of 65.50% and an AuPRCr of 31.83 within a 1.2m region, while maintaining competitive semantic occupancy prediction performance and generalization in real-world urban driving scenes. The established datasets and source code will be made publicly available at https://github.com/7uHeng/OccOoD.