Universality for the conjugate gradient and MINRES algorithms on sample covariance matrices

TL;DR

This paper proves a universality property and a central limit theorem for the residual norms of conjugate gradient and MINRES algorithms applied to sample covariance matrices, revealing predictable iteration behavior.

Contribution

It introduces a probabilistic analysis establishing universality and a CLT for Krylov methods on sample covariance matrices, extending understanding of their spectral properties.

Findings

Residual norms follow a Gaussian distribution asymptotically.

Iteration counts are nearly deterministic due to the CLT.

Universality holds for a broad class of matrices satisfying moment conditions.

Abstract

We present a probabilistic analysis of two Krylov subspace methods for solving linear systems. We prove a central limit theorem for norms of the residual vectors that are produced by the conjugate gradient and MINRES algorithms when applied to a wide class of sample covariance matrices satisfying some standard moment conditions. The proof involves establishing a four moment theorem for the so-called spectral measure, implying, in particular, universality for the matrix produced by the Lanczos iteration. The central limit theorem then implies an almost-deterministic iteration count for the iterative methods in question.