Adaptive novelty detection with false discovery rate guarantee

TL;DR

This paper introduces AdaDetect, a flexible semi-supervised novelty detection method that guarantees false discovery rate control without distributional assumptions, adaptable to various classifiers and data scenarios.

Contribution

AdaDetect is a novel, adaptable framework that controls FDR in novelty detection, leveraging conformal inference and multiple testing techniques, with variants for null proportion adaptation.

Findings

AdaDetect controls FDR in finite samples across different datasets.

The method adapts to data characteristics, improving detection power.

Applications include synthetic data and astrophysics datasets.

Abstract

This paper studies the semi-supervised novelty detection problem where a set of "typical" measurements is available to the researcher. Motivated by recent advances in multiple testing and conformal inference, we propose AdaDetect, a flexible method that is able to wrap around any probabilistic classification algorithm and control the false discovery rate (FDR) on detected novelties in finite samples without any distributional assumption other than exchangeability. In contrast to classical FDR-controlling procedures that are often committed to a pre-specified p-value function, AdaDetect learns the transformation in a data-adaptive manner to focus the power on the directions that distinguish between inliers and outliers. Inspired by the multiple testing literature, we further propose variants of AdaDetect that are adaptive to the proportion of nulls while maintaining the finite-sample FDR control. The methods are illustrated on synthetic datasets and real-world datasets, including an application in astrophysics.