Revisiting Energy-Based Model for Out-of-Distribution Detection

TL;DR

This paper proposes a novel out-of-distribution detection framework called OEST* that leverages simple data transformations to generate peripheral-distribution data, improving energy-based models' ability to distinguish in-distribution from OOD samples.

Contribution

The paper introduces OEST*, a new energy-based OOD detection method using simple transformations for data augmentation, providing a theoretically grounded energy barrier for better separation.

Findings

OEST* outperforms or matches state-of-the-art methods across benchmarks.

The energy barrier concept enhances OOD detection robustness.

Empirical results validate the effectiveness of simple transformation-generated data.

Abstract

Out-of-distribution (OOD) detection is an essential approach to robustifying deep learning models, enabling them to identify inputs that fall outside of their trained distribution. Existing OOD detection methods usually depend on crafted data, such as specific outlier datasets or elaborate data augmentations. While this is reasonable, the frequent mismatch between crafted data and OOD data limits model robustness and generalizability. In response to this issue, we introduce Outlier Exposure by Simple Transformations (OEST), a framework that enhances OOD detection by leveraging "peripheral-distribution" (PD) data. Specifically, PD data are samples generated through simple data transformations, thus providing an efficient alternative to manually curated outliers. We adopt energy-based models (EBMs) to study PD data. We recognize the "energy barrier" in OOD detection, which characterizes the energy difference between in-distribution (ID) and OOD samples and eases detection. PD data are introduced to establish the energy barrier during training. Furthermore, this energy barrier concept motivates a theoretically grounded energy-barrier loss to replace the classical energy-bounded loss, leading to an improved paradigm, OEST*, which achieves a more effective and theoretically sound separation between ID and OOD samples. We perform empirical validation of our proposal, and extensive experiments across various benchmarks demonstrate that OEST* achieves better or similar accuracy compared with state-of-the-art methods.