foap4: Adaptive mesh refinement with OpenACC, MPI, and p4est

TL;DR

This paper presents foap4, a Fortran-based adaptive mesh refinement framework utilizing OpenACC, MPI, and p4est, demonstrating efficient GPU-accelerated simulations of gas dynamics in 2D and 3D.

Contribution

The paper introduces a new Fortran framework for AMR that effectively leverages GPUs with OpenACC and MPI, enabling efficient large-scale simulations.

Findings

AMR simulations are efficient on GPUs with OpenACC and MPI.

Small grid blocks of 8^3 or 16^3 cells still yield good performance.

Benchmarks include solving Euler's equations in 2D and 3D.

Abstract

GPUs and other accelerators are increasingly used for scientific computing. In the future, we want to add GPU support to parallel adaptive mesh refinement (AMR) codes written in Fortran. To understand which changes are necessary to obtain good performance we have developed foap4, an AMR framework implemented in Fortran that uses OpenACC, MPI, and the p4est library. We discuss the design and implementation of the framework. Several benchmark problems are considered, in which Euler's equations of gas dynamics are solved using explicit time integration. These benchmarks are performed in both 2D and 3D, using static and adaptive meshes, for varying problem sizes on different hardware. Our results show that AMR simulations can be carried out efficiently on GPUs with OpenACC and MPI, even when using relatively small grid blocks of $8^3$ or $16^3$ cells.