Unbiased Risk Estimation in the Normal Means Problem via Coupled Bootstrap Techniques

TL;DR

This paper introduces a coupled bootstrap method for unbiased risk estimation in the normal means problem, which works without assumptions on the estimator and relates to SURE, providing accurate risk estimates through synthetic noise and averaging.

Contribution

It presents a novel coupled bootstrap approach that yields unbiased risk estimates for any estimator in the normal means problem, connecting to SURE and analyzing bias-variance trade-offs.

Findings

CB estimator is unbiased under no assumptions

It recovers SURE under certain conditions

Performs well in simulated experiments

Abstract

We develop a new approach for estimating the risk of an arbitrary estimator of the mean vector in the classical normal means problem. The key idea is to generate two auxiliary data vectors, by adding carefully constructed normal noise vectors to the original data. We then train the estimator of interest on the first auxiliary vector and test it on the second. In order to stabilize the risk estimate, we average this procedure over multiple draws of the synthetic noise vector. A key aspect of this coupled bootstrap (CB) approach is that it delivers an unbiased estimate of risk under no assumptions on the estimator of the mean vector, albeit for a modified and slightly "harder" version of the original problem, where the noise variance is elevated. We prove that, under the assumptions required for the validity of Stein's unbiased risk estimator (SURE), a limiting version of the CB estimator recovers SURE exactly. We then analyze a bias-variance decomposition of the error of the CB estimator, which elucidates the effects of the variance of the auxiliary noise and the number of bootstrap samples on the accuracy of the estimator. Lastly, we demonstrate that the CB estimator performs favorably in various simulated experiments.