Some facts about the optimality of the LSE in the Gaussian sequence model with convex constraint

TL;DR

This paper investigates the conditions under which the least squares estimator (LSE) is minimax optimal in a convex constrained Gaussian sequence model, linking optimality to the local Gaussian width's properties and providing theoretical algorithms for risk assessment.

Contribution

It characterizes the necessary and sufficient conditions for LSE optimality in convex constrained Gaussian models, connecting it to the Lipschitz property of local Gaussian widths and analyzing various set examples.

Findings

LSE is minimax optimal when local Gaussian width mapping is Lipschitz.

Optimality depends on geometric properties of the constraint set.

Algorithms can estimate worst-case risk for different convex sets.

Abstract

We consider a convex constrained Gaussian sequence model and characterize necessary and sufficient conditions for the least squares estimator (LSE) to be minimax optimal. For a closed convex set $K\subset \mathbb{R}^n$ we observe $Y=\mu+\xi$ for $\xi\sim \mathcal{N}(0,\sigma^2\mathbb{I}_n)$ and $\mu\in K$ and aim to estimate $\mu$. We characterize the worst case risk of the LSE in multiple ways by analyzing the behavior of the local Gaussian width on $K$. We demonstrate that optimality is equivalent to a Lipschitz property of the local Gaussian width mapping. We also provide theoretical algorithms that search for the worst case risk. We then provide examples showing optimality or suboptimality of the LSE on various sets, including $\ell_p$ balls for $p\in[1,2]$, pyramids, solids of revolution, and multivariate isotonic regression, among others.