DPU: Dynamic Prototype Updating for Multimodal Out-of-Distribution Detection

TL;DR

This paper introduces DPU, a novel framework that dynamically updates class prototypes to improve multimodal out-of-distribution detection by accounting for intra-class variability, significantly enhancing detection performance.

Contribution

The paper presents a new plug-and-play method, DPU, that adaptively updates class centers based on intra-class variance, improving robustness in multimodal OOD detection.

Findings

DPU achieves up to 80% improvement in Far-OOD detection.

It outperforms existing methods across multiple datasets and algorithms.

DPU enhances model robustness by accounting for intra-class variability.

Abstract

Out-of-distribution (OOD) detection is essential for ensuring the robustness of machine learning models by identifying samples that deviate from the training distribution. While traditional OOD detection has primarily focused on single-modality inputs, such as images, recent advances in multimodal models have demonstrated the potential of leveraging multiple modalities (e.g., video, optical flow, audio) to enhance detection performance. However, existing methods often overlook intra-class variability within in-distribution (ID) data, assuming that samples of the same class are perfectly cohesive and consistent. This assumption can lead to performance degradation, especially when prediction discrepancies are uniformly amplified across all samples. To address this issue, we propose Dynamic Prototype Updating (DPU), a novel plug-and-play framework for multimodal OOD detection that accounts for intra-class variations. Our method dynamically updates class center representations for each class by measuring the variance of similar samples within each batch, enabling adaptive adjustments. This approach allows us to amplify prediction discrepancies based on the updated class centers, thereby improving the model's robustness and generalization across different modalities. Extensive experiments on two tasks, five datasets, and nine base OOD algorithms demonstrate that DPU significantly improves OOD detection performance, setting a new state-of-the-art in multimodal OOD detection, with improvements of up to 80 percent in Far-OOD detection. To facilitate accessibility and reproducibility, our code is publicly available on GitHub.