GPU-Accelerated DNS of Compressible Turbulent Flows

TL;DR

This paper presents a GPU-accelerated version of a CFD solver for compressible turbulent flows, achieving significant speedups and demonstrating scalability on supercomputers for large-scale turbulence simulations.

Contribution

The paper introduces a GPU-optimized implementation of HyPar, enabling high-resolution turbulence simulations on exascale heterogeneous platforms with demonstrated scalability.

Findings

200x speedup of key kernels on GPU

Successful strong and weak scaling on NVIDIA V100 GPUs

Simulation of turbulence with up to 1024^3 grid points on 1024 GPUs

Abstract

This paper explores strategies to transform an existing CPU-based high-performance computational fluid dynamics solver, HyPar, for compressible flow simulations on emerging exascale heterogeneous (CPU+GPU) computing platforms. The scientific motivation for developing a GPU-enhanced version of HyPar is to simulate canonical turbulent flows at the highest resolution possible on such platforms. We show that optimizing memory operations and thread blocks results in 200x speedup of computationally intensive kernels compared with a CPU core. Using multiple GPUs and CUDA-aware MPI communication, we demonstrate both strong and weak scaling of our GPU-based HyPar implementation on the NVIDIA Volta V100 GPUs. We simulate the decay of homogeneous isotropic turbulence in a triply periodic box on grids with up to $1024^3$ points (5.3 billion degrees of freedom) and on up to 1,024 GPUs. We compare the wall times for CPU-only and CPU+GPU simulations. The results presented in the paper are obtained on the Summit and Lassen supercomputers at Oak Ridge and Lawrence Livermore National Laboratories, respectively.