A highly scalable approach to solving linear systems using two-stage multisplitting

TL;DR

This paper introduces a scalable hybrid method for solving large sparse linear systems by partitioning matrices into blocks, using local solvers within blocks, and coupling them with multisplitting techniques to reduce global communication, demonstrated on high-performance computing systems.

Contribution

The paper presents a novel two-stage multisplitting approach that improves scalability of iterative linear solvers by combining local optimized solvers with a block coupling strategy, reducing global communication.

Findings

Demonstrated scalability up to 32768 cores on a Cray XE6 system.

Hybrid approach reduces the need for global communication in large-scale systems.

Conventional solvers outperform the hybrid at current scales, but trends indicate potential advantages at higher core counts.

Abstract

Iterative methods for solving large sparse systems of linear equations are widely used in many HPC applications. Extreme scaling of these methods can be difficult, however, since global communication to form dot products is typically required at every iteration. To try to overcome this limitation we propose a hybrid approach, where the matrix is partitioned into blocks. Within each block, we use a highly optimised (parallel) conventional solver, but we then couple the blocks together using block Jacobi or some other multisplitting technique that can be implemented in either a synchronous or an asynchronous fashion. This allows us to limit the block size to the point where the conventional iterative methods no longer scale, and to avoid global communication (and possibly synchronisation) across all processes. Our block framework has been built to use PETSc, a popular scientific suite for solving sparse linear systems, as the synchronous intra-block solver, and we demonstrate results on up to 32768 cores of a Cray XE6 system. At this scale, the conventional solvers are still more efficient, though trends suggest that the hybrid approach may be beneficial at higher core counts.