Learning Non-Linear Invariants for Unsupervised Out-of-Distribution Detection

TL;DR

This paper introduces a novel framework using a normalizing flow-like architecture to learn non-linear invariants for unsupervised out-of-distribution detection, significantly improving performance on large-scale benchmarks.

Contribution

It extends the affine invariants approach to non-linear invariants with a new learnable architecture, achieving state-of-the-art results and applicability to tabular data.

Findings

Achieves state-of-the-art results on U-OOD benchmarks

Demonstrates effectiveness on tabular data

Maintains desirable properties of affine invariants

Abstract

The inability of deep learning models to handle data drawn from unseen distributions has sparked much interest in unsupervised out-of-distribution (U-OOD) detection, as it is crucial for reliable deep learning models. Despite considerable attention, theoretically-motivated approaches are few and far between, with most methods building on top of some form of heuristic. Recently, U-OOD was formalized in the context of data invariants, allowing a clearer understanding of how to characterize U-OOD, and methods leveraging affine invariants have attained state-of-the-art results on large-scale benchmarks. Nevertheless, the restriction to affine invariants hinders the expressiveness of the approach. In this work, we broaden the affine invariants formulation to a more general case and propose a framework consisting of a normalizing flow-like architecture capable of learning non-linear invariants. Our novel approach achieves state-of-the-art results on an extensive U-OOD benchmark, and we demonstrate its further applicability to tabular data. Finally, we show our method has the same desirable properties as those based on affine invariants.