The radius of statistical efficiency

TL;DR

This paper introduces the radius of statistical efficiency (RSE), a measure of robustness for estimation problems, linking it to the Fisher information matrix and problem complexity across various models.

Contribution

We define and compute the RSE, a new robustness measure, and establish its reciprocal relationship with problem complexity, extending spectral function theory of measures.

Findings

RSE quantifies robustness by smallest data perturbation causing Fisher matrix singularity.

RSE inversely correlates with problem complexity, paralleling classical matrix theorems.

Develops spectral measure theory extending eigenvalue optimization results.

Abstract

Classical results in asymptotic statistics show that the Fisher information matrix controls the difficulty of estimating a statistical model from observed data. In this work, we introduce a companion measure of robustness of an estimation problem: the radius of statistical efficiency (RSE) is the size of the smallest perturbation to the problem data that renders the Fisher information matrix singular. We compute RSE up to numerical constants for a variety of testbed problems, including principal component analysis, generalized linear models, phase retrieval, bilinear sensing, and matrix completion. Interestingly, we observe a precise reciprocal relationship between RSE and the intrinsic complexity/sensitivity of the problem instance, paralleling the classical Eckart-Young theorem in numerical analysis. To establish our results, we develop theory for spectral functions of measures that extends well-known results from matrix analysis and eigenvalue optimization$-$a contribution that may be of interest beyond our immediate findings.