Matrix Means and a Novel High-Dimensional Shrinkage Phenomenon

TL;DR

This paper explores the use of the matrix harmonic mean as an alternative to the arithmetic mean in high-dimensional matrix estimation, revealing its advantages and limitations through theoretical analysis and simulations.

Contribution

It introduces the high-dimensional benefits of the matrix harmonic mean, connects it to shrinkage estimators, and proposes a new nonlinear shrinkage method.

Findings

Harmonic mean improves estimation error in high dimensions.

Improvement in operator norm does not guarantee better eigenvector recovery.

A new nonlinear shrinkage estimator is derived from Rao-Blackwellization.

Abstract

Many statistical settings call for estimating a population parameter, most typically the population mean, based on a sample of matrices. The most natural estimate of the population mean is the arithmetic mean, but there are many other matrix means that may behave differently, especially in high dimensions. Here we consider the matrix harmonic mean as an alternative to the arithmetic matrix mean. We show that in certain high-dimensional regimes, the harmonic mean yields an improvement over the arithmetic mean in estimation error as measured by the operator norm. Counter-intuitively, studying the asymptotic behavior of these two matrix means in a spiked covariance estimation problem, we find that this improvement in operator norm error does not imply better recovery of the leading eigenvector. We also show that a Rao-Blackwellized version of the harmonic mean is equivalent to a linear shrinkage estimator studied previously in the high-dimensional covariance estimation literature, while applying a similar Rao-Blackwellization to regularized sample covariance matrices yields a novel nonlinear shrinkage estimator. Simulations complement the theoretical results, illustrating the conditions under which the harmonic matrix mean yields an empirically better estimate.