Taming False Positives in Out-of-Distribution Detection with Human Feedback

TL;DR

This paper introduces a mathematically grounded framework for out-of-distribution detection that uses expert feedback to dynamically control false positive rates, ensuring safety in critical applications.

Contribution

It proposes a novel OOD detection method that guarantees FPR constraints are met while minimizing human feedback, adaptable to any scoring function.

Findings

Maintains FPR at most 5% in experiments

Maximizes TPR under FPR constraints

Works with any OOD scoring function

Abstract

Robustness to out-of-distribution (OOD) samples is crucial for safely deploying machine learning models in the open world. Recent works have focused on designing scoring functions to quantify OOD uncertainty. Setting appropriate thresholds for these scoring functions for OOD detection is challenging as OOD samples are often unavailable up front. Typically, thresholds are set to achieve a desired true positive rate (TPR), e.g., $95\%$ TPR. However, this can lead to very high false positive rates (FPR), ranging from 60 to 96\%, as observed in the Open-OOD benchmark. In safety-critical real-life applications, e.g., medical diagnosis, controlling the FPR is essential when dealing with various OOD samples dynamically. To address these challenges, we propose a mathematically grounded OOD detection framework that leverages expert feedback to \emph{safely} update the threshold on the fly. We provide theoretical results showing that it is guaranteed to meet the FPR constraint at all times while minimizing the use of human feedback. Another key feature of our framework is that it can work with any scoring function for OOD uncertainty quantification. Empirical evaluation of our system on synthetic and benchmark OOD datasets shows that our method can maintain FPR at most $5\%$ while maximizing TPR.