Parallelized Domain Decomposition for Multi-Dimensional Lagrangian Random Walk, Mass-Transfer Particle Tracking Schemes

TL;DR

This paper introduces a parallelized domain decomposition strategy for multi-dimensional Lagrangian particle tracking methods, optimizing tiling procedures to significantly accelerate reactive transport simulations on high-performance computing systems.

Contribution

It develops and analyzes a multi-dimensional domain decomposition approach for mass-transfer particle tracking, extending 1D methods to 2D and 3D with optimal tiling strategies based on physical and computational parameters.

Findings

Nearly perfect linear speedup in 2D up to 2700 cores

Significant reduction of simulation time from 5 hours to 8 seconds in 2D

Maintains speedup up to 1700 cores in 3D

Abstract

We develop a multi-dimensional, parallelized domain decomposition strategy (DDC) for mass-transfer particle tracking (MTPT) methods. These methods are a type of Lagrangian algorithm for simulating reactive transport and are able to be parallelized by employing large numbers of CPU cores to accelerate run times. In this work, we investigate different procedures for "tiling" the domain in two and three dimensions, (2-d and 3-d), as this type of formal DDC construction is currently limited to 1-d. An optimal tiling is prescribed based on physical problem parameters and the number of available CPU cores, as each tiling provides distinct results in both accuracy and run time. We further extend the most efficient technique to 3-d for comparison, leading to an analytical discussion of the effect of dimensionality on strategies for implementing DDC schemes. Increasing computational resources (cores) within the DDC method produces a trade-off between inter-node communication and on-node work. For an optimally subdivided diffusion problem, the 2-d parallelized algorithm achieves nearly perfect linear speedup in comparison with the serial run up to around 2700 cores, reducing a 5-hour simulation to 8 seconds, and the 3-d algorithm maintains appreciable speedup up to 1700 cores.