BOLT: Block-Orthonormal Lanczos for Trace estimation of matrix functions

TL;DR

This paper introduces BOLT, a new method for trace estimation of large matrices that works efficiently with limited memory and partial access, improving accuracy and applicability over existing methods like Hutch++ and SLQ.

Contribution

BOLT offers a simpler, more memory-efficient alternative to Hutch++, with theoretical guarantees and superior performance in flat-spectrum regimes, especially under partial matrix access constraints.

Findings

BOLT matches Hutch++ accuracy with simpler implementation.

Subblock SLQ enables trace estimation with limited matrix access.

Empirical results show strong performance in high-dimensional settings.

Abstract

Efficient matrix trace estimation is essential for scalable computation of log-determinants, matrix norms, and distributional divergences. In many large-scale applications, the matrices involved are too large to store or access in full, making even a single matrix-vector (mat-vec) product infeasible. Instead, one often has access only to small subblocks of the matrix or localized matrix-vector products on restricted index sets. Hutch++ achieves optimal convergence rate but relies on randomized SVD and assumes full mat-vec access, making it difficult to apply in these constrained settings. We propose the Block-Orthonormal Stochastic Lanczos Quadrature (BOLT), which matches Hutch++ accuracy with a simpler implementation based on orthonormal block probes and Lanczos iterations. BOLT builds on the Stochastic Lanczos Quadrature (SLQ) framework, which combines random probing with Krylov subspace methods to efficiently approximate traces of matrix functions, and performs better than Hutch++ in near flat-spectrum regimes. To address memory limitations and partial access constraints, we introduce Subblock SLQ, a variant of BOLT that operates only on small principal submatrices. As a result, this framework yields a proxy KL divergence estimator and an efficient method for computing the Wasserstein-2 distance between Gaussians - both compatible with low-memory and partial-access regimes. We provide theoretical guarantees and demonstrate strong empirical performance across a range of high-dimensional settings.