Improving Out-of-Distribution Detection via Dynamic Covariance Calibration

TL;DR

This paper introduces a dynamic covariance calibration method that improves out-of-distribution detection by adjusting prior geometry in real-time, effectively handling distorted data distributions and enhancing detection accuracy across multiple models.

Contribution

The paper proposes a novel dynamic covariance updating technique that refines prior geometry in response to new data, addressing limitations of static methods in OOD detection.

Findings

Significantly improves OOD detection performance across various models.

Effective on both CIFAR and ImageNet-1k datasets.

Enhances detection accuracy of pre-trained models including self-supervised ones.

Abstract

Out-of-Distribution (OOD) detection is essential for the trustworthiness of AI systems. Methods using prior information (i.e., subspace-based methods) have shown effective performance by extracting information geometry to detect OOD data with a more appropriate distance metric. However, these methods fail to address the geometry distorted by ill-distributed samples, due to the limitation of statically extracting information geometry from the training distribution. In this paper, we argue that the influence of ill-distributed samples can be corrected by dynamically adjusting the prior geometry in response to new data. Based on this insight, we propose a novel approach that dynamically updates the prior covariance matrix using real-time input features, refining its information. Specifically, we reduce the covariance along the direction of real-time input features and constrain adjustments to the residual space, thus preserving essential data characteristics and avoiding effects on unintended directions in the principal space. We evaluate our method on two pre-trained models for the CIFAR dataset and five pre-trained models for ImageNet-1k, including the self-supervised DINO model. Extensive experiments demonstrate that our approach significantly enhances OOD detection across various models. The code is released at https://github.com/workerbcd/ooddcc.