A cooperative conjugate gradient method for linear systems permitting multithread implementation of low complexity

TL;DR

This paper introduces a cooperative multithreaded conjugate gradient method that updates multiple directions simultaneously, reducing complexity for large symmetric positive definite systems and demonstrating practical efficiency on multicore computers.

Contribution

It generalizes the conjugate gradient method by allowing matrix-based step sizes and implements a multithreaded cooperative approach for improved performance.

Findings

Multithread implementation achieves worst-case complexity of O(n^{2+1/3})

Numerical experiments confirm theoretical complexity benefits

Cooperative approach enhances parallel efficiency in large-scale systems

Abstract

This paper proposes a generalization of the conjugate gradient (CG) method used to solve the equation $Ax=b$ for a symmetric positive definite matrix $A$ of large size $n$. The generalization consists of permitting the scalar control parameters (= stepsizes in gradient and conjugate gradient directions) to be replaced by matrices, so that multiple descent and conjugate directions are updated simultaneously. Implementation involves the use of multiple agents or threads and is referred to as cooperative CG (cCG), in which the cooperation between agents resides in the fact that the calculation of each entry of the control parameter matrix now involves information that comes from the other agents. For a sufficiently large dimension $n$, the use of an optimal number of cores gives the result that the multithread implementation has worst case complexity $O(n^{2+1/3})$ in exact arithmetic. Numerical experiments, that illustrate the interest of theoretical results, are carried out on a multicore computer.