Mixed Precision General Alternating-Direction Implicit Method for Solving Large Sparse Linear Systems

TL;DR

This paper presents a mixed precision GADI method that accelerates large sparse linear system solutions by solving subsystems in low precision and residuals in high precision, with convergence analysis and GPU performance validation.

Contribution

It introduces a three-precision GADI scheme with convergence guarantees, a GPR-based parameter selection strategy, and demonstrates significant speedups on large-scale problems.

Findings

Achieved up to 3.1x speedup on large-scale problems.

Convergence of mixed precision GADI is theoretically established.

Validated performance improvements on NVIDIA A100 GPU.

Abstract

In this article, we introduce a three-precision formulation of the General Alternating-Direction Implicit method (GADI) designed to accelerate the solution of large-scale sparse linear systems $Ax=b$. GADI is a framework that can represent many existing Alternating-Direction Implicit (ADI) methods. These methods are a class of linear solvers based on a splitting of $A$ such that the solution of the original linear system can be decomposed into the successive computation of easy-to-solve structured subsystems. Our proposed mixed precision scheme for GADI solves these subsystems in low precision to reduce the overall execution time while computing the residual and solution update in high precision to enable the solution to converge to high accuracy. We develop a rounding error analysis of mixed precision GADI that establishes the rates of convergence of the forward and backward errors to certain limiting accuracies. Our analysis also highlights the conditions on the splitting matrices under which mixed precision GADI is guaranteed to converge for a given set of precisions. We then discuss a systematic and robust strategy for selecting the GADI regularization parameter $\alpha$, whose adjustment is critical for performance. Specifically, our proposed strategy makes use of a Gaussian Process Regression (GPR) model trained on a dataset of low-dimensional problems to initialize $\alpha$. Finally, we proceed to a performance analysis of mixed precision GADI on an NVIDIA A100 GPU to validate our approach. Using low precision (Bfloat16 or FP32) to solve the subsystems, we obtain speedups of $2.6\times$, $1.7\times$, and $3.1\times$ over a full double precision GADI implementation on large-scale 2D, 3D convection-diffusion and complex reaction-diffusion problems (up to $1.3\times 10^{8}$ unknowns), respectively.