Stage-parallel fully implicit Runge-Kutta implementations with optimal multilevel preconditioners at the scaling limit

TL;DR

This paper introduces a highly scalable, stage-parallel implementation of fully implicit Runge-Kutta methods with optimal multilevel preconditioners, demonstrating significant performance gains at the scaling limit for large-scale heat problems.

Contribution

It develops a novel stage-parallel preconditioning approach for Radau IIA methods, enabling efficient large-scale parallel computations with detailed performance analysis.

Findings

Achieves higher throughputs near the scaling limit with large process counts.

Speedup increases linearly with the number of stages, bounded by the number of stages.

Performance model and studies confirm scalability on up to 150,000 processes.

Abstract

We present an implementation of a fully stage-parallel preconditioner for Radau IIA type fully implicit Runge--Kutta methods, which approximates the inverse of $A_Q$ from the Butcher tableau by the lower triangular matrix resulting from an LU decomposition and diagonalizes the system with as many blocks as stages. For the transformed system, we employ a block preconditioner where each block is distributed and solved by a subgroup of processes in parallel. For combination of partial results, we either use a communication pattern resembling Cannon's algorithm or shared memory. A performance model and a large set of performance studies (including strong scaling runs with up to 150k processes on 3k compute nodes) conducted for a time-dependent heat problem, using matrix-free finite element methods, indicate that the stage-parallel implementation can reach higher throughputs when the block solvers operate at lower parallel efficiencies, which occurs near the scaling limit. Achievable speedup increases linearly with number of stages and are bounded by the number of stages. Furthermore, we show that the presented stage-parallel concepts are also applicable to the case that $A_Q$ is directly diagonalized, which requires complex arithmetic or the solution of two-by-two blocks and sequentializes parts of the algorithm. Alternatively to distributing stages and assigning them to distinct processes, we discuss the possibility of batching operations from different stages together.