A Functional Data Perspective and Baseline On Multi-Layer Out-of-Distribution Detection

TL;DR

This paper introduces a novel functional data perspective for multi-layer out-of-distribution detection, leveraging sample trajectories across layers to improve detection accuracy beyond traditional methods.

Contribution

It proposes a baseline method based on functional anomaly detection that exploits the entire layer-wise trajectories, requiring no special architecture or supervision.

Findings

Effective OOD detection on computer vision benchmarks

Outperforms several state-of-the-art baselines

Utilizes sample trajectories for improved detection

Abstract

A key feature of out-of-distribution (OOD) detection is to exploit a trained neural network by extracting statistical patterns and relationships through the multi-layer classifier to detect shifts in the expected input data distribution. Despite achieving solid results, several state-of-the-art methods rely on the penultimate or last layer outputs only, leaving behind valuable information for OOD detection. Methods that explore the multiple layers either require a special architecture or a supervised objective to do so. This work adopts an original approach based on a functional view of the network that exploits the sample's trajectories through the various layers and their statistical dependencies. It goes beyond multivariate features aggregation and introduces a baseline rooted in functional anomaly detection. In this new framework, OOD detection translates into detecting samples whose trajectories differ from the typical behavior characterized by the training set. We validate our method and empirically demonstrate its effectiveness in OOD detection compared to strong state-of-the-art baselines on computer vision benchmarks.