Smoothed Analysis for the Conjugate Gradient Algorithm

TL;DR

This paper analyzes the convergence rate of the conjugate gradient algorithm under random matrix perturbations using smoothed analysis, providing bounds and moment estimates that enhance understanding of its performance in practical scenarios.

Contribution

It introduces a rigorous smoothed analysis framework for the conjugate gradient algorithm with random perturbations from the Laguerre ensemble, including finite moment estimates of halting times.

Findings

Derived bounds on convergence rates under random perturbations.

Estimated all finite moments of halting times in the critical regime.

Validated theoretical results with numerical experiments.

Abstract

The purpose of this paper is to establish bounds on the rate of convergence of the conjugate gradient algorithm when the underlying matrix is a random positive definite perturbation of a deterministic positive definite matrix. We estimate all finite moments of a natural halting time when the random perturbation is drawn from the Laguerre unitary ensemble in a critical scaling regime explored in Deift et al. (2016). These estimates are used to analyze the expected iteration count in the framework of smoothed analysis, introduced by Spielman and Teng (2001). The rigorous results are compared with numerical calculations in several cases of interest.