Enhancing OOD Detection Using Latent Diffusion

TL;DR

This paper introduces a novel latent space-based outlier synthesis framework, Outlier-Aware Learning (OAL), combined with contrastive learning and knowledge distillation, to improve OOD detection efficiency and accuracy with less computational cost.

Contribution

It proposes a new latent space OOD data generation method, along with contrastive learning and distillation modules, enhancing OOD detection performance and robustness.

Findings

Outperforms existing methods on benchmark datasets

Requires fewer outliers and less computational resources

Improves ID/OOD discrimination accuracy

Abstract

Out-of-distribution (OOD) detection is crucial for the reliable deployment of machine learning models in real-world scenarios, enabling the identification of unknown samples or objects. A prominent approach to enhance OOD detection performance involves leveraging auxiliary datasets for training. Recent efforts have explored using generative models, such as Stable Diffusion (SD), to synthesize outlier data in the pixel space. However, synthesizing OOD data in the pixel space can lead to reduced robustness due to over-generation. To address this challenge, we propose Outlier-Aware Learning (OAL), a novel framework that generates synthetic OOD training data within the latent space, taking a further step to study how to utilize Stable Diffusion for developing a latent-based outlier synthesis approach. This improvement facilitates network training with fewer outliers and less computational cost. Besides, to regularize the model's decision boundary, we develop a mutual information-based contrastive learning module (MICL) that amplifies the distinction between In-Distribution (ID) and collected OOD data. Moreover, we develop a knowledge distillation module to prevent the degradation of ID classification accuracy when training with OOD data. The superior performance of our method on several benchmark datasets demonstrates its efficiency and effectiveness. Source code is available in https://github.com/HengGao12/OAL.