Conformal novelty detection with false discovery rate control at the boundary

TL;DR

This paper extends the support line correction method to conformal novelty detection, addressing false discovery rate control issues near decision boundaries, and proposes new procedures with proven bFDR control validated by experiments.

Contribution

It introduces novel conformal-specific alternatives to the SL method that provably control boundary FDR, improving reliability in novelty detection.

Findings

Proposed methods control bFDR in conformal setting

Numerical experiments validate theoretical guarantees

New procedures outperform existing approaches near thresholds

Abstract

Conformal novelty detection is a classical machine learning task for which uncertainty quantification is essential for providing reliable results. Recent work has shown that the BH procedure applied to conformal p-values controls the false discovery rate (FDR). Unfortunately, the BH procedure can lead to over-optimistic assessments near the rejection threshold, with an increase of false discoveries at the margin as pointed out by Soloff et al. (2024). This issue is solved therein by the support line (SL) correction, which is proven to control the boundary false discovery rate (bFDR) in the independent, non-conformal setting. The present work extends the SL method to the conformal setting: first, we show that the SL procedure can violate the bFDR control in this specific setting. Second, we propose several alternatives that provably control the bFDR in the conformal setting. Finally, numerical experiments with both synthetic and real data support our theoretical findings and show the relevance of the new proposed procedures.