DAVIS: OOD Detection via Dominant Activations and Variance for Increased Separation

TL;DR

The paper introduces DAVIS, a post-hoc method that enhances out-of-distribution detection by incorporating channel-wise variance and dominant activations, significantly improving performance across multiple architectures.

Contribution

DAVIS leverages overlooked activation statistics to improve OOD detection, providing a simple, effective, and broadly applicable enhancement over existing methods.

Findings

DAVIS achieves a 48.26% reduction in FPR95 on CIFAR-10 with ResNet-18.

It improves OOD detection performance on CIFAR-100 and ImageNet-1k benchmarks.

Incorporating variance and maximum activations enhances the discriminative power of feature representations.

Abstract

Detecting out-of-distribution (OOD) inputs is a critical safeguard for deploying machine learning models in the real world. However, most post-hoc detection methods operate on penultimate feature representations derived from global average pooling (GAP) -- a lossy operation that discards valuable distributional statistics from activation maps prior to global average pooling. We contend that these overlooked statistics, particularly channel-wise variance and dominant (maximum) activations, are highly discriminative for OOD detection. We introduce DAVIS, a simple and broadly applicable post-hoc technique that enriches feature vectors by incorporating these crucial statistics, directly addressing the information loss from GAP. Extensive evaluations show DAVIS sets a new benchmark across diverse architectures, including ResNet, DenseNet, and EfficientNet. It achieves significant reductions in the false positive rate (FPR95), with improvements of 48.26\% on CIFAR-10 using ResNet-18, 38.13\% on CIFAR-100 using ResNet-34, and 26.83\% on ImageNet-1k benchmarks using MobileNet-v2. Our analysis reveals the underlying mechanism for this improvement, providing a principled basis for moving beyond the mean in OOD detection.