Conjugate Gradient Methods are Not Efficient: Experimental Study of the Locality Limitation

TL;DR

This paper experimentally demonstrates that Conjugate Gradient methods are limited by graph locality, which restricts information flow and imposes a lower bound on the number of iterations needed for accurate solutions.

Contribution

It provides an empirical analysis of the locality limitation in Conjugate Gradient methods, highlighting the impact of graph diameter on convergence.

Findings

Conjugate Gradient convergence is limited by graph diameter.

Information transfer in CG is restricted by the sparsity pattern.

A minimum number of iterations is necessary due to locality constraints.

Abstract

The convergence of the Conjugate Gradient method is subject to a locality limitation which imposes a lower bound on the number of iterations required before a qualitatively accurate approximation can be obtained. This limitation originates from the restricted transport of information in the graph induced by the sparsity pattern of the system matrix. In each iteration, information from the right-hand side can propagate only across directly connected graph nodes. The diameter of this graph therefore determines a minimum number of iterations that is necessary to achieve an acceptable level of accuracy.