Scaling the memory wall using mixed-precision -- HPG-MxP on an exascale machine

TL;DR

This paper demonstrates a highly optimized implementation of the HPG-MxP benchmark on an exascale system, achieving a 1.6x speedup using mixed-precision algorithms for sparse matrix computations on GPU supercomputers.

Contribution

It presents the first practical implementation of the HPG-MxP benchmark on an exascale system with algorithm enhancements and reports significant speedup using mixed-precision techniques.

Findings

Achieved 1.6x speedup with mixed-precision on GPU supercomputers.

Optimized implementation of HPG-MxP benchmark for exascale systems.

Demonstrated practical benefits of mixed-precision in memory bandwidth limited applications.

Abstract

Mixed-precision algorithms have been proposed as a way for scientific computing to benefit from some of the gains seen for artificial intelligence (AI) on recent high performance computing (HPC) platforms. A few applications dominated by dense matrix operations have seen substantial speedups by utilizing low precision formats such as FP16. However, a majority of scientific simulation applications are memory bandwidth limited. Beyond preliminary studies, the practical gain from using mixed-precision algorithms on a given HPC system is largely unclear. The High Performance GMRES Mixed Precision (HPG-MxP) benchmark has been proposed to measure the useful performance of a HPC system on sparse matrix-based mixed-precision applications. In this work, we present a highly optimized implementation of the HPG-MxP benchmark for an exascale system and describe our algorithm enhancements. We show for the first time a speedup of 1.6x using a combination of double- and single-precision on modern GPU-based supercomputers.