Exploring Dynamic Load Balancing Algorithms for Block-Structured Mesh-and-Particle Simulations in AMReX

TL;DR

This paper evaluates various dynamic load balancing algorithms for large-scale mesh-and-particle simulations in HPC, highlighting the effectiveness of Knapsack and painter's partition algorithms under certain conditions.

Contribution

It introduces and compares novel load balancing algorithms, including a painter's partition-based SFC and a combined Knapsack+SFC approach, for improved HPC simulation efficiency.

Findings

Knapsack and painter's partition algorithms outperform AMReX's strategy with limited weight deviation.

Performance benefits diminish as weight variety increases.

Brute-force evaluation provides insights into algorithm efficiency.

Abstract

Load balancing is critical for successful large-scale high-performance computing (HPC) simulations. With modern supercomputers increasing in complexity and variability, dynamic load balancing is becoming more critical to use computational resources efficiently. In this study, performed during a summer collaboration at Lawrence Berkeley National Laboratory, we investigate various standard dynamic load-balancing algorithms. This includes the time evaluation of a brute-force solve for application in algorithmic evaluation, as well as quality and time evaluations of the Knapsack algorithm, an SFC algorithm, and two novel algorithms: a painter's partition-based SFC algorithm and a combination Knapsack+SFC methodology-based on hardware topology. The results suggest Knapsack and painter's partition-based algorithms should be among the first algorithms evaluated by HPC codes for cases with limited weight deviation and will perform at least slightly better than AMReX's percentage-tracking partitioning strategy across most simulations, although effects diminish as weight variety increases.