Two Variations on the XTrace Algorithm

TL;DR

This paper explores two modifications to the XTrace algorithm for matrix trace estimation, including variance reduction and low-rank approximation, with experiments showing mixed practical benefits and spectrum-dependent improvements.

Contribution

It introduces two novel variations of XTrace, one for variance reduction and another for low-rank approximation, enhancing trace estimation accuracy depending on matrix properties.

Findings

Variance reduction offers slight practical benefits.

Low-rank approximation can significantly improve estimates.

Performance depends on the matrix spectrum.

Abstract

This paper studies two potential modifications of XTrace (Epperly et al., SIMAX 45(1):1-23, 2024), a randomized algorithm for estimating the trace of a matrix. The first is a variance reduction step that averages the output of XTrace over right-multiplications of the test vectors by random orthogonal matrices. The second is to form a low-rank approximation to the matrix using the whole Krylov space produced by the test vectors, rather than the output of a single power iteration as is used by XTrace. Experiments on synthetic data show that the first modification offers only slight benefits in practice, while the second can lead to significant improvements depending on the spectrum of the matrix.