In-Memory Load Balancing for Discontinuous Galerkin Methods on Polytopal Meshes

TL;DR

This paper introduces an in-memory load balancing method for high-order discontinuous Galerkin simulations on complex polytopal meshes, improving efficiency and scalability on heterogeneous hardware.

Contribution

It presents a lightweight, system-agnostic load balancing strategy that dynamically redistributes mesh elements during simulations, enhancing performance on complex geometries.

Findings

Significant efficiency recovery on heterogeneous meshes.

Maintains strong and weak scaling performance.

Effective in large-scale parallel simulations.

Abstract

High-order accurate discontinuous Galerkin (DG) methods have emerged as powerful tools for solving partial differential equations such as the compressible Navier-Stokes equations due to their excellent dispersion-dissipation properties and scalability on modern hardware. The open-source DG framework FLEXI has recently been extended to support DG schemes on general polytopal elements including tetrahedra, prisms, and pyramids. This advancement enables simulations on complex geometries where purely hexahedral meshes are difficult or impossible to generate. However, the use of meshes with heterogeneous element types introduces a workload imbalance, a consequence of the temporal evolution of modal rather than nodal degrees of freedom and the accompanying transformations. In this work, we present a lightweight, system-agnostic in-memory load balancing strategy designed for high-order DG solvers. The method employs high-precision runtime measurements and efficient data redistribution to dynamically reassign mesh elements along a space-filling curve. We demonstrate the effectiveness of the approach through simulations of the Taylor-Green vortex and large-scale parallel runs on the EuroHPC pre-exascale system MareNostrum 5. Results show that the proposed strategy recovers a significant fraction of the lost efficiency on heterogeneous meshes while retaining excellent strong and weak scaling.