Signal-to-noise ratio aware minimaxity and higher-order asymptotics

TL;DR

This paper introduces a signal-to-noise ratio aware minimax framework that improves the theoretical understanding and practical estimation of sparse signals by incorporating SNR considerations and higher-order asymptotics.

Contribution

It proposes a new SNR-aware minimax approach and demonstrates how higher-order asymptotics can refine risk approximations and estimator discovery.

Findings

SNR-aware minimax framework better explains empirical estimator performance.

Higher-order asymptotics provide accurate risk approximations.

Guides the development of improved sparse signal estimators.

Abstract

Since its development, the minimax framework has been one of the corner stones of theoretical statistics, and has contributed to the popularity of many well-known estimators, such as the regularized M-estimators for high-dimensional problems. In this paper, we will first show through the example of sparse Gaussian sequence model, that the theoretical results under the classical minimax framework are insufficient for explaining empirical observations. In particular, both hard and soft thresholding estimators are (asymptotically) minimax, however, in practice they often exhibit sub-optimal performances at various signal-to-noise ratio (SNR) levels. The first contribution of this paper is to demonstrate that this issue can be resolved if the signal-to-noise ratio is taken into account in the construction of the parameter space. We call the resulting minimax framework the signal-to-noise ratio aware minimaxity. The second contribution of this paper is to showcase how one can use higher-order asymptotics to obtain accurate approximations of the SNR-aware minimax risk and discover minimax estimators. The theoretical findings obtained from this refined minimax framework provide new insights and practical guidance for the estimation of sparse signals.