Revisiting Logit Distributions for Reliable Out-of-Distribution Detection

TL;DR

This paper introduces LogitGap, a novel post-hoc method that leverages relationships within model logits to improve out-of-distribution detection, achieving state-of-the-art results across various benchmarks.

Contribution

The paper proposes LogitGap, a new approach that explicitly exploits logit relationships and a training-free strategy to identify informative logits for better OOD detection.

Findings

LogitGap outperforms existing methods on multiple benchmarks.

The approach is effective across vision-language and vision-only models.

The method is training-free and easy to deploy.

Abstract

Out-of-distribution (OOD) detection is critical for ensuring the reliability of deep learning models in open-world applications. While post-hoc methods are favored for their efficiency and ease of deployment, existing approaches often underexploit the rich information embedded in the model's logits space. In this paper, we propose LogitGap, a novel post-hoc OOD detection method that explicitly exploits the relationship between the maximum logit and the remaining logits to enhance the separability between in-distribution (ID) and OOD samples. To further improve its effectiveness, we refine LogitGap by focusing on a more compact and informative subset of the logit space. Specifically, we introduce a training-free strategy that automatically identifies the most informative logits for scoring. We provide both theoretical analysis and empirical evidence to validate the effectiveness of our approach. Extensive experiments on both vision-language and vision-only models demonstrate that LogitGap consistently achieves state-of-the-art performance across diverse OOD detection scenarios and benchmarks. Code is available at https://github.com/GIT-LJc/LogitGap.