Improving the Performance of the GMRES Method using Mixed-Precision Techniques

TL;DR

This paper demonstrates that mixed-precision techniques in GMRES can significantly enhance performance by reducing data movement, while maintaining double-precision accuracy, through selective use of single and double precision in computations.

Contribution

The study introduces a mixed-precision GMRES implementation that retains accuracy and achieves up to 24% performance gains, using generic programming and parallel libraries.

Findings

Mixed-precision GMRES retains double-precision accuracy.

Performance improved by up to 24% with mixed-precision.

Only residual and solution updates need double precision.

Abstract

The GMRES method is used to solve sparse, non-symmetric systems of linear equations arising from many scientific applications. The solver performance within a single node is memory bound, due to the low arithmetic intensity of its computational kernels. To reduce the amount of data movement, and thus, to improve performance, we investigated the effect of using a mix of single and double precision while retaining double-precision accuracy. Previous efforts have explored reduced precision in the preconditioner, but the use of reduced precision in the solver itself has received limited attention. We found that GMRES only needs double precision in computing the residual and updating the approximate solution to achieve double-precision accuracy, although it must restart after each improvement of single-precision accuracy. This finding holds for the tested orthogonalization schemes: Modified Gram-Schmidt (MGS) and Classical Gram-Schmidt with Re-orthogonalization (CGSR). Furthermore, our mixed-precision GMRES, when restarted at least once, performed 19% and 24% faster on average than double-precision GMRES for MGS and CGSR, respectively. Our implementation uses generic programming techniques to ease the burden of coding implementations for different data types. Our use of the Kokkos library allowed us to exploit parallelism and optimize data management. Additionally, KokkosKernels was used when producing performance results. In conclusion, using a mix of single and double precision in GMRES can improve performance while retaining double-precision accuracy.