Outlier Synthesis via Hamiltonian Monte Carlo for Out-of-Distribution Detection

TL;DR

This paper introduces HamOS, a novel outlier synthesis framework using Hamiltonian Monte Carlo, which efficiently generates diverse outliers from in-distribution data to improve OOD detection.

Contribution

The paper proposes a Hamiltonian Monte Carlo-based method for synthesizing outliers that overcomes quality and cost issues of previous approaches, enhancing OOD detection.

Findings

HamOS generates diverse outliers effectively.

HamOS achieves competitive performance on benchmarks.

The framework is efficient with high sampling acceptance rates.

Abstract

Out-of-distribution (OOD) detection is crucial for developing trustworthy and reliable machine learning systems. Recent advances in training with auxiliary OOD data demonstrate efficacy in enhancing detection capabilities. Nonetheless, these methods heavily rely on acquiring a large pool of high-quality natural outliers. Some prior methods try to alleviate this problem by synthesizing virtual outliers but suffer from either poor quality or high cost due to the monotonous sampling strategy and the heavy-parameterized generative models. In this paper, we overcome all these problems by proposing the Hamiltonian Monte Carlo Outlier Synthesis (HamOS) framework, which views the synthesis process as sampling from Markov chains. Based solely on the in-distribution data, the Markov chains can extensively traverse the feature space and generate diverse and representative outliers, hence exposing the model to miscellaneous potential OOD scenarios. The Hamiltonian Monte Carlo with sampling acceptance rate almost close to 1 also makes our framework enjoy great efficiency. By empirically competing with SOTA baselines on both standard and large-scale benchmarks, we verify the efficacy and efficiency of our proposed HamOS.