Improved bounds on sample size for implicit matrix trace estimators

TL;DR

This paper improves theoretical bounds on the number of samples needed for Monte Carlo trace estimators of implicit matrices, considering various distributions and matrix properties, with experiments validating the bounds.

Contribution

It provides new, tighter bounds on sample size for trace estimation methods, extending and refining previous theoretical results for different random vector distributions.

Findings

Improved bound on sample size for Hutchinson method, removing rank dependence.

New bounds for Gaussian and unit vector estimators based on matrix properties.

Necessary bounds for Gaussian estimator highlighting conditions for effectiveness.

Abstract

This article is concerned with Monte-Carlo methods for the estimation of the trace of an implicitly given matrix $A$ whose information is only available through matrix-vector products. Such a method approximates the trace by an average of $N$ expressions of the form $\ww^t (A\ww)$, with random vectors $\ww$ drawn from an appropriate distribution. We prove, discuss and experiment with bounds on the number of realizations $N$ required in order to guarantee a probabilistic bound on the relative error of the trace estimation upon employing Rademacher (Hutchinson), Gaussian and uniform unit vector (with and without replacement) probability distributions. In total, one necessary bound and six sufficient bounds are proved, improving upon and extending similar estimates obtained in the seminal work of Avron and Toledo (2011) in several dimensions. We first improve their bound on $N$ for the Hutchinson method, dropping a term that relates to $rank(A)$ and making the bound comparable with that for the Gaussian estimator. We further prove new sufficient bounds for the Hutchinson, Gaussian and the unit vector estimators, as well as a necessary bound for the Gaussian estimator, which depend more specifically on properties of the matrix $A$. As such they may suggest for what type of matrices one distribution or another provides a particularly effective or relatively ineffective stochastic estimation method.