Analyzing the effect of local rounding error propagation on the maximal attainable accuracy of the pipelined Conjugate Gradient method

TL;DR

This paper investigates how local rounding errors affect the maximum achievable accuracy of pipelined Conjugate Gradient methods and proposes a residual replacement strategy to mitigate accuracy loss while maintaining parallel efficiency.

Contribution

It introduces an analysis of rounding error propagation in pipelined CG and proposes an automated residual replacement to enhance accuracy without sacrificing performance.

Findings

Rounding errors significantly impact pipelined CG accuracy.

The residual replacement strategy effectively improves attainable accuracy.

Numerical results confirm the method's effectiveness across benchmarks.

Abstract

Pipelined Krylov subspace methods typically offer improved strong scaling on parallel HPC hardware compared to standard Krylov subspace methods for large and sparse linear systems. In pipelined methods the traditional synchronization bottleneck is mitigated by overlapping time-consuming global communications with useful computations. However, to achieve this communication hiding strategy, pipelined methods introduce additional recurrence relations for a number of auxiliary variables that are required to update the approximate solution. This paper aims at studying the influence of local rounding errors that are introduced by the additional recurrences in the pipelined Conjugate Gradient method. Specifically, we analyze the impact of local round-off effects on the attainable accuracy of the pipelined CG algorithm and compare to the traditional CG method. Furthermore, we estimate the gap between the true residual and the recursively computed residual used in the algorithm. Based on this estimate we suggest an automated residual replacement strategy to reduce the loss of attainable accuracy on the final iterative solution. The resulting pipelined CG method with residual replacement improves the maximal attainable accuracy of pipelined CG, while maintaining the efficient parallel performance of the pipelined method. This conclusion is substantiated by numerical results for a variety of benchmark problems.