High-probability minimax lower bounds

TL;DR

This paper introduces the concept of minimax quantiles to better understand the tail behaviour of risks in statistical estimation, developing high-probability bounds and applying them to various problems for more detailed difficulty assessments.

Contribution

It develops high-probability variants of classical minimax lower bound methods and introduces minimax quantiles as a finer measure of statistical problem difficulty.

Findings

Recovered recent results in robust mean estimation and stochastic convex optimization.

Derived new bounds in covariance matrix estimation and sparse linear regression.

Provided a unified framework for analyzing tail risks in diverse statistical problems.

Abstract

The minimax risk is often considered as a gold standard against which we can compare specific statistical procedures. Nevertheless, as has been observed recently in robust and heavy-tailed estimation problems, the inherent reduction of the (random) loss to its expectation may entail a significant loss of information regarding its tail behaviour. In an attempt to avoid such a loss, we introduce the notion of a minimax quantile, and seek to articulate its dependence on the quantile level. To this end, we develop high-probability variants of the classical Le Cam and Fano methods, as well as a technique to convert local minimax risk lower bounds to lower bounds on minimax quantiles. To illustrate the power of our framework, we deploy our techniques on several examples, recovering recent results in robust mean estimation and stochastic convex optimisation, as well as obtaining several new results in covariance matrix estimation, sparse linear regression, nonparametric density estimation and isotonic regression. Our overall goal is to argue that minimax quantiles can provide a finer-grained understanding of the difficulty of statistical problems, and that, in wide generality, lower bounds on these quantities can be obtained via user-friendly tools.