A GPU-Accelerated Sharp Interface Immersed Boundary Solver for Large Scale Flow Simulations

TL;DR

This paper introduces a GPU-accelerated sharp-interface immersed boundary solver, ViCar3D, enabling large-scale 3D flow simulations with significant speedup and high scalability on multi-GPU systems.

Contribution

The work presents the implementation and acceleration of ViCar3D on GPUs using OpenACC, CUDA Fortran, and MPI, achieving high efficiency and scalability for large flow problems.

Findings

20X speedup over CPU implementation for flow past a rectangular wing

Capable of simulating 200 million mesh points on a single node with four GPUs

Achieved 92% and 93% maximum scaling efficiencies for strong and weak scaling tests

Abstract

Immersed boundary methods (IBMs) facilitate the simulation of flows around stationary, moving, and deforming bodies on Cartesian grids. However, extending these simulations to the large grid sizes required for realistic flow problems remains a significant computational challenge. In this work, we present the implementation and acceleration of ViCar3D, a sharp-interface immersed boundary solver, on graphical processing units (GPUs). We utilize OpenACC, CUDA Fortran and MPI to reprogram \emph{ViCar3D}, a sharp-interface immersed boundary solver, on multi-GPU architectures. Verification and scalability studies are performed for two benchmark cases: two-dimensional flow past a circular cylinder and direct numerical simulation (DNS) of flow past a finite rectangular wing. For the latter, we observe an approximately 20X speedup (node-to-node comparison) relative to the CPU-based implementation. The GPU-accelerated solver is capable of simulating complex 3D flows with up to 200 million mesh points on a single node equipped with four GPUs. Strong and weak scaling tests demonstrate maximum scaling efficiencies of 92\% and 93\%, respectively, on multi-GPU systems. We further test the code to simulate fluid flow past complex-shaped single-body and multi-body cases.