Extremely Simple Multimodal Outlier Synthesis for Out-of-Distribution Detection and Segmentation

TL;DR

This paper introduces Feature Mixing, a simple and fast multimodal outlier synthesis method with theoretical backing, improving OOD detection and segmentation across various datasets and modalities.

Contribution

We propose Feature Mixing, a modality-agnostic technique for multimodal outlier synthesis, along with CARLA-OOD, a new dataset for OOD segmentation, advancing state-of-the-art performance.

Findings

Achieves state-of-the-art OOD detection performance

Provides 10x to 370x speedup over previous methods

Demonstrates effectiveness across multiple datasets and modalities

Abstract

Out-of-distribution (OOD) detection and segmentation are crucial for deploying machine learning models in safety-critical applications such as autonomous driving and robot-assisted surgery. While prior research has primarily focused on unimodal image data, real-world applications are inherently multimodal, requiring the integration of multiple modalities for improved OOD detection. A key challenge is the lack of supervision signals from unknown data, leading to overconfident predictions on OOD samples. To address this challenge, we propose Feature Mixing, an extremely simple and fast method for multimodal outlier synthesis with theoretical support, which can be further optimized to help the model better distinguish between in-distribution (ID) and OOD data. Feature Mixing is modality-agnostic and applicable to various modality combinations. Additionally, we introduce CARLA-OOD, a novel multimodal dataset for OOD segmentation, featuring synthetic OOD objects across diverse scenes and weather conditions. Extensive experiments on SemanticKITTI, nuScenes, CARLA-OOD datasets, and the MultiOOD benchmark demonstrate that Feature Mixing achieves state-of-the-art performance with a $10 \times$ to $370 \times$ speedup. Our source code and dataset will be available at https://github.com/mona4399/FeatureMixing.