GPU accelerated MHD in the DISPATCH framework using directive-based programming

TL;DR

This paper demonstrates a GPU-accelerated MHD solver integrated into the DISPATCH framework using OpenMP directives, achieving significant speedups and maintaining accuracy across tests.

Contribution

It introduces a novel GPU implementation of an MHD solver within DISPATCH using directive-based programming, enabling efficient offloading and substantial performance gains.

Findings

7.3x speed-up on mini-app with GPU vs CPU

9.8x overall speedup in large-scale tests

GPU routine is two orders of magnitude faster than CPU core

Abstract

We present a GPU-accelerated implementation of a magnetohydrodynamic (MHD) solver using directive-based programming with OpenMP target offloading. The solver is integrated into the DISPATCH framework, which organises the computational domain into a collection of asynchronously updated patches. To reduce GPU kernel launch overhead, patches are grouped into "bunches" that are updated collectively. While porting the particular solver required a complete code refactoring, it yielded performance gains on both GPU and CPU. A stand-alone mini-app achieved a 7.3x speed-up compared to a single NVIDIA A100 GPU to seven AMD 7F72 Rome CPU cores. Within the full DISPATCH framework, the GPU-accelerated MHD Bunch solver showed excellent agreement with the CPU-based reference implementation on standard test problems such as the Sod shock tube and Orszag-Tang vortex. In large-scale 3D tests, the GPU implementation achieved a 9.8x overall speedup, comparing one GPU to 12 CPU cores, with the core MHD update routine being two orders of magnitude faster on the GPU than on a single CPU core. These results demonstrate that OpenMP offloading can provide substantial performance improvements for astrophysical codes while maintaining portability and accuracy. The work also demonstrates how new codes should be structured to allow simple and efficient directive-based GPU offloading.