Conformalized Method for Empirical Bayes Normal Mean Inference Problem with Heteroscedastic Variance

TL;DR

This paper introduces COIN, a conformal inference method for normal mean testing that maintains false discovery rate control without relying on correct prior specification or estimation, unlike traditional empirical Bayes methods.

Contribution

The paper proposes COIN, a novel conformal inference algorithm that ensures valid inference in normal mean problems regardless of prior misspecification or estimation errors.

Findings

COIN asymptotically controls the false discovery rate at the nominal level.

Theoretical guarantees are provided for data-splitting strategies when external data are unavailable.

Numerical studies and real data examples demonstrate COIN's effectiveness.

Abstract

We study the normal mean inference problem, which involves simultaneous testing of the means of many normal distributions. This problem has been extensively studied within the empirical Bayes (EB) framework. However, the reliability of most EB methods heavily depends on two key conditions: (i) the prior distribution is correctly specified, and (ii) it can be accurately estimated. In practice, both conditions are difficult to satisfy, and it is often unclear whether they hold in a given application. To overcome these limitations, we propose a new algorithm, called COIN (COnformal Inference for Normal mean inference problem). Unlike traditional empirical Bayes approaches, COIN produces decision rules whose validity does not depend on the correct specification or accurate estimation of the prior. We theoretically prove that COIN asymptotically controls the false discovery rate at the nominal level, even in the presence of prior misspecification or estimation errors. Since the COIN algorithm requires an external training dataset to estimate the prior distribution and conformity score function, we introduce two data-splitting strategies -- sample-splitting and feature-splitting -- for the case where such external data are unavailable. We provide theoretical guarantees for the data-splitting strategies and demonstrate their effectiveness through extensive numerical studies and three real data examples.