Probability Density from Latent Diffusion Models for Out-of-Distribution Detection

TL;DR

This paper investigates the effectiveness of likelihood-based out-of-distribution detection using latent diffusion models in the representation space of a pre-trained ResNet-18, addressing limitations observed in pixel space.

Contribution

It demonstrates that density estimation in the representation space can improve OOD detection, challenging previous doubts about likelihood methods in generative models.

Findings

Likelihood-based OOD detection performs better in representation space.

Representation space density estimation can outperform pixel space methods.

The study provides insights into the role of feature spaces in OOD detection.

Abstract

Despite rapid advances in AI, safety remains the main bottleneck to deploying machine-learning systems. A critical safety component is out-of-distribution detection: given an input, decide whether it comes from the same distribution as the training data. In generative models, the most natural OOD score is the data likelihood. Actually, under the assumption of uniformly distributed OOD data, the likelihood is even the optimal OOD detector, as we show in this work. However, earlier work reported that likelihood often fails in practice, raising doubts about its usefulness. We explore whether, in practice, the representation space also suffers from the inability to learn good density estimation for OOD detection, or if it is merely a problem of the pixel space typically used in generative models. To test this, we trained a Variational Diffusion Model not on images, but on the representation space of a pre-trained ResNet-18 to assess the performance of our likelihood-based detector in comparison to state-of-the-art methods from the OpenOOD suite.