Efficient Parallelization for AMR MHD Multiphysics Calculations; Implementation in AstroBEAR

TL;DR

This paper presents new parallelization and memory management techniques for AMR MHD multiphysics simulations, implemented in AstroBEAR, to enhance scalability and efficiency on large compute systems.

Contribution

The authors introduce a threading approach for grid advances and a distributed tree algorithm for memory management in AMR simulations, improving parallelization and scalability.

Findings

Enhanced parallelization across physical space and AMR levels.

Improved load balancing through threading of grid advances.

Reduced memory overhead with distributed tree structure.

Abstract

Current AMR simulations require algorithms that are highly parallelized and manage memory efficiently. As compute engines grow larger, AMR simulations will require algorithms that achieve new levels of efficient parallelization and memory management. We have attempted to employ new techniques to achieve both of these goals. Patch or grid based AMR often employs ghost cells to decouple the hyperbolic advances of each grid on a given refinement level. This decoupling allows each grid to be advanced independently. In AstroBEAR we utilize this independence by threading the grid advances on each level with preference going to the finer level grids. This allows for global load balancing instead of level by level load balancing and allows for greater parallelization across both physical space and AMR level. Threading of level advances can also improve performance by interleaving communication with computation, especially in deep simulations with many levels of refinement. To improve memory management we have employed a distributed tree algorithm that requires processors to only store and communicate local sections of the AMR tree structure with neighboring processors.