Improved Spectral Density Estimation via Explicit and Implicit Deflation

TL;DR

This paper introduces an improved spectral density estimation method combining deflation and polynomial moment matching, achieving stronger error bounds especially for matrices with fast singular value decay, and explains the effectiveness of the SLQ method.

Contribution

It proposes a novel spectral density estimation algorithm with explicit deflation, providing tighter error bounds and analyzing the implicit deflation in SLQ, a popular existing method.

Findings

Achieves $ ilde{O}( ext{ell} \, ext{log} n + 1/\epsilon)$ matrix-vector products for error bounds.

Shows that SLQ matches the derived bounds despite not using explicit deflation.

Provides nearly tight lower bounds on the number of matrix-vector products needed for certain error levels.

Abstract

We study algorithms for approximating the spectral density of a symmetric matrix $A$ that is accessed through matrix-vector product queries. By combining a previously studied Chebyshev polynomial moment matching method with a deflation step that approximately projects off the largest magnitude eigendirections of $A$ before estimating the spectral density, we give an $\epsilon\cdot\sigma_\ell(A)$ error approximation to the spectral density in the Wasserstein-$1$ metric using $O(\ell\log n+ 1/\epsilon)$ matrix-vector products, where $\sigma_\ell(A)$ is the $\ell^{th}$ largest singular value of $A$. In the common case when $A$ exhibits fast singular value decay, our bound can be much stronger than prior work, which gives an error bound of $\epsilon \cdot ||A||_2$ using $O(1/\epsilon)$ matrix-vector products. We also show that it is nearly tight: any algorithm giving error $\epsilon \cdot \sigma_\ell(A)$ must use $\Omega(\ell+1/\epsilon)$ matrix-vector products. We further show that the popular Stochastic Lanczos Quadrature (SLQ) method matches the above bound, even though SLQ itself is parameter-free and performs no explicit deflation. This bound explains the strong practical performance of SLQ, and motivates a simple variant of SLQ that achieves an even tighter error bound. Our error bound for SLQ leverages an analysis that views it as an implicit polynomial moment matching method, along with recent results on low-rank approximation with single-vector Krylov methods. We use these results to show that the method can perform implicit deflation as part of moment matching.