A Conformalized Empirical Bayes Method for Multiple Testing with Side Information

TL;DR

This paper introduces CLAW, a novel conformalized empirical Bayes method that effectively incorporates side information for multiple testing, offering finite-sample FDR control and improved performance over existing approaches.

Contribution

The paper develops CLAW, a new conformalized empirical Bayes framework that controls FDR in finite samples while integrating auxiliary covariates for enhanced testing accuracy.

Findings

CLAW controls FDR in finite samples under weaker conditions.

CLAW outperforms existing methods in simulated and real data.

The approach effectively incorporates side information for multiple testing.

Abstract

This article presents a Conformalized Locally Adaptive Weighting (CLAW) approach to multiple testing with side information. The proposed method employs innovative data-driven strategies to construct pairwise exchangeable scores, which are integrated into a generic algorithm that leverages a mirror process for controlling the false discovery rate (FDR). By combining principles from empirical Bayes with powerful techniques in conformal inference, CLAW provides a valid and efficient framework for incorporating structural information from both test data and auxiliary covariates. Unlike existing empirical Bayes FDR methods that primarily offer asymptotic validity, often under strong regularity conditions, CLAW controls the FDR in finite samples under weaker conditions. Extensive numerical studies using both simulated and real data demonstrate that CLAW exhibits superior performance compared to existing methods.