Mitigating the Likelihood Paradox in Flow-based OOD Detection via Entropy Manipulation

TL;DR

This paper proposes a method to reduce the likelihood paradox in flow-based out-of-distribution detection by manipulating input entropy based on semantic similarity, improving detection performance without retraining models.

Contribution

It introduces a novel entropy manipulation technique that enhances OOD detection in flow-based models without additional training, supported by theoretical analysis and empirical results.

Findings

Improves AUROC scores on standard OOD benchmarks

Theoretically increases the likelihood gap between in-distribution and OOD samples

Works without retraining the density model

Abstract

Deep generative models that can tractably compute input likelihoods, including normalizing flows, often assign unexpectedly high likelihoods to out-of-distribution (OOD) inputs. We mitigate this likelihood paradox by manipulating input entropy based on semantic similarity, applying stronger perturbations to inputs that are less similar to an in-distribution memory bank. We provide a theoretical analysis showing that entropy control increases the expected log-likelihood gap between in-distribution and OOD samples in favor of the in-distribution, and we explain why the procedure works without any additional training of the density model. We then evaluate our method against likelihood-based OOD detectors on standard benchmarks and find consistent AUROC improvements over baselines, supporting our explanation.