On optimality of Bayesian testimation in the normal means problem

TL;DR

This paper develops a Bayesian framework for sparse high-dimensional normal means problems, leading to estimators that unify and extend existing thresholding and model selection methods with proven optimality.

Contribution

It introduces a family of $l_0$-type penalties derived from prior distributions, providing a flexible Bayesian approach with optimality guarantees in sparse settings.

Findings

Bayesian estimators achieve minimax optimality under mild prior conditions.

The framework encompasses many known thresholding procedures as special cases.

Certain priors lead to adaptively optimal estimators for various sparse sequences.

Abstract

We consider a problem of recovering a high-dimensional vector $\mu$ observed in white noise, where the unknown vector $\mu$ is assumed to be sparse. The objective of the paper is to develop a Bayesian formalism which gives rise to a family of $l_0$-type penalties. The penalties are associated with various choices of the prior distributions $\pi_n(\cdot)$ on the number of nonzero entries of $\mu$ and, hence, are easy to interpret. The resulting Bayesian estimators lead to a general thresholding rule which accommodates many of the known thresholding and model selection procedures as particular cases corresponding to specific choices of $\pi_n(\cdot)$. Furthermore, they achieve optimality in a rather general setting under very mild conditions on the prior. We also specify the class of priors $\pi_n(\cdot)$ for which the resulting estimator is adaptively optimal (in the minimax sense) for a wide range of sparse sequences and consider several examples of such priors.