The conjugate gradient method with various viewpoints

TL;DR

This paper surveys the conjugate gradient (CG) method's connections with various computational methods, explores polynomial relationships, and reexamines convergence rates from new perspectives, providing a comprehensive theoretical overview.

Contribution

It offers a unified view of CG's relationships with other methods and introduces new insights into its convergence behavior from alternative viewpoints.

Findings

Connections between CG and other methods are systematically reviewed.

Residual and conjugate vector polynomials are analyzed for matrix information.

Convergence rates are reconsidered from a novel perspective.

Abstract

Connections of the conjugate gradient (CG) method with other methods in computational mathematics are surveyed, including the connections with the conjugate direction method, the subspace optimization method and the quasi-Newton method BFGS in numrical optimization, and the Lanczos method in numerical linear algebra. Two sequences of polynomials related to residual vectors and conjugate vectors are reviewed, where the residual polynomials are similar to orthogonal polynomials in the approximation theory and the roots of the polynomials reveal certain information of the coefficient matrix. The convergence rates of the steepest descent and CG are reconsidered in a viewpoint different from textbooks. The connection of infinite dimensional CG with finite dimensional preconditioned CG is also reviewed via numerical solution of an elliptic equation.