GPU-Accelerated Discontinuous Galerkin Methods: 30x Speedup on 345 Billion Unknowns

TL;DR

This paper presents a GPU-accelerated discontinuous Galerkin method for fluid dynamics that achieves a 30x speedup over CPU implementations and demonstrates excellent scalability on thousands of GPUs for extremely large problems.

Contribution

It introduces a GPU-accelerated discontinuous Galerkin method using OCCA for performance portability, achieving significant speedup and scalability for large-scale fluid dynamics simulations.

Findings

30x speedup over CPU implementations

Scalability up to 6,144 GPUs for 345 billion unknowns

CUDA-Aware MPI communication improves performance by 24%

Abstract

A discontinuous Galerkin method for the discretization of the compressible Euler equations, the governing equations of inviscid fluid dynamics, on Cartesian meshes is developed for use of Graphical Processing Units via OCCA, a unified approach to performance portability on multi-threaded hardware architectures. A 30x time-to-solution speedup over CPU-only implementations using non-CUDA-Aware MPI communications is demonstrated up to 1,536 NVIDIA V100 GPUs and parallel strong scalability is shown up to 6,144 NVIDIA V100 GPUs for a problem containing 345 billion unknowns. A comparison of CUDA-Aware MPI communication to non-GPUDirect communication is performed demonstrating an additional 24% speedup on eight nodes composed of 32 NVIDIA V100 GPUs.