Monte Carlo Methods for Estimating the Diagonal of a Real Symmetric Matrix

TL;DR

This paper introduces Monte Carlo estimators for efficiently approximating the diagonal of real symmetric matrices using probabilistic bounds, with analysis applicable to various random vectors and an application to sensitivity metrics.

Contribution

It presents new probabilistic bounds for Monte Carlo estimators of matrix diagonals, utilizing matrix concentration inequalities and analyzing different random vector types.

Findings

Bounds mostly independent of matrix dimension

Estimators improve with diagonal dominance

Sparse Rademacher vectors perform poorly in practice

Abstract

For real symmetric matrices that are accessible only through matrix vector products, we present Monte Carlo estimators for computing the diagonal elements. Our probabilistic bounds for normwise absolute and relative errors apply to Monte Carlo estimators based on random Rademacher, sparse Rademacher, normalized and unnormalized Gaussian vectors, and to vectors with bounded fourth moments. The novel use of matrix concentration inequalities in our proofs represents a systematic model for future analyses. Our bounds mostly do not depend on the matrix dimension, target different error measures than existing work, and imply that the accuracy of the estimators increases with the diagonal dominance of the matrix. An application to derivative-based global sensitivity metrics corroborates this, as do numerical experiments on synthetic test matrices. We recommend against the use in practice of sparse Rademacher vectors, which are the basis for many randomized sketching and sampling algorithms, because they tend to deliver barely a digit of accuracy even under large sampling amounts.