Conditional Testing based on Localized Conformal p-values

TL;DR

This paper introduces localized conformal p-values for conditional testing, enabling effective outlier detection, label screening, and distribution comparison with theoretical guarantees and demonstrated superior performance.

Contribution

It proposes a new framework for conditional testing using localized conformal p-values, with applications to outlier detection, label screening, and two-sample tests, supported by theoretical analysis and empirical validation.

Findings

Finite-sample FDR control in outlier detection

FWER control in label screening

Superior performance in simulations and real data

Abstract

In this paper, we address conditional testing problems through the conformal inference framework. We define the localized conformal p-values by inverting prediction intervals and prove their theoretical properties. These defined p-values are then applied to several conditional testing problems to illustrate their practicality. Firstly, we propose a conditional outlier detection procedure to test for outliers in the conditional distribution with finite-sample false discovery rate (FDR) control. We also introduce a novel conditional label screening problem with the goal of screening multivariate response variables and propose a screening procedure to control the family-wise error rate (FWER). Finally, we consider the two-sample conditional distribution test and define a weighted U-statistic through the aggregation of localized p-values. Numerical simulations and real-data examples validate the superior performance of our proposed strategies.