High-dimensional Location Estimation via Norm Concentration for Subgamma Vectors

TL;DR

This paper develops finite-sample bounds for location estimation using smoothed estimators, extending previous work to high dimensions and providing new bounds for subgamma vectors, approaching the Cramér-Rao limit asymptotically.

Contribution

It introduces a finite-sample analysis for high-dimensional location estimation using smoothed estimators and proves a novel bound on high-dimensional subgamma vectors.

Findings

Finite-sample bounds for location estimation with smoothed estimators.

Extension of theory to high-dimensional distributions.

New bound on the norm of high-dimensional subgamma vectors.

Abstract

In location estimation, we are given $n$ samples from a known distribution $f$ shifted by an unknown translation $\lambda$, and want to estimate $\lambda$ as precisely as possible. Asymptotically, the maximum likelihood estimate achieves the Cram\'er-Rao bound of error $\mathcal N(0, \frac{1}{n\mathcal I})$, where $\mathcal I$ is the Fisher information of $f$. However, the $n$ required for convergence depends on $f$, and may be arbitrarily large. We build on the theory using \emph{smoothed} estimators to bound the error for finite $n$ in terms of $\mathcal I_r$, the Fisher information of the $r$-smoothed distribution. As $n \to \infty$, $r \to 0$ at an explicit rate and this converges to the Cram\'er-Rao bound. We (1) improve the prior work for 1-dimensional $f$ to converge for constant failure probability in addition to high probability, and (2) extend the theory to high-dimensional distributions. In the process, we prove a new bound on the norm of a high-dimensional random variable whose 1-dimensional projections are subgamma, which may be of independent interest.