A Tight Analysis of Hutchinson's Diagonal Estimator

TL;DR

This paper provides a tighter analysis of Hutchinson's diagonal estimator, showing it converges with a bound independent of matrix size, improving previous results by removing a logarithmic factor.

Contribution

The paper offers a refined theoretical bound for Hutchinson's estimator, eliminating the dependence on the matrix dimension n in the error bound.

Findings

Estimator's error bound is independent of matrix size n.

The bound improves previous results by a logarithmic factor.

Convergence rate depends on the Frobenius norm of the off-diagonal elements.

Abstract

Let $\mathbf{A}\in \mathbb{R}^{n\times n}$ be a matrix with diagonal $\text{diag}(\mathbf{A})$ and let $\bar{\mathbf{A}}$ be $\mathbf{A}$ with its diagonal set to all zeros. We show that Hutchinson's estimator run for $m$ iterations returns a diagonal estimate $\tilde{d}\in \mathbb{R}^n$ such that with probability $(1-\delta)$, $$\|\tilde{d} - \text{diag}(\mathbf{A})\|_2 \leq c\sqrt{\frac{\log(2/\delta)}{m}}\|\bar{\mathbf{A}}\|_F,$$ where $c$ is a fixed constant independent of all other parameters. This results improves on a recent result of [Baston and Nakatsukasa, 2022] by a $\log(n)$ factor, yielding a bound that is independent of the matrix dimension $n$.