Parallel grid library for rapid and flexible simulation development

TL;DR

This paper introduces dccrg, a flexible, scalable grid library for parallel simulations supporting adaptive mesh refinement and asynchronous data transfer, enabling efficient high-performance computing in fluid and particle simulations.

Contribution

The paper presents dccrg, a novel grid library that supports adaptive mesh refinement and asynchronous communication, improving scalability and flexibility for parallel simulation development.

Findings

Scales efficiently up to 32,000 cores in magnetohydrodynamic tests.

Supports arbitrary C++ classes as cell data.

Achieves good scalability with some limitations in AMR at high process counts.

Abstract

We present an easy to use and flexible grid library for developing highly scalable parallel simulations. The distributed cartesian cell-refinable grid (dccrg) supports adaptive mesh refinement and allows an arbitrary C++ class to be used as cell data. The amount of data in grid cells can vary both in space and time allowing dccrg to be used in very different types of simulations, for example in fluid and particle codes. Dccrg transfers the data between neighboring cells on different processes transparently and asynchronously allowing one to overlap computation and communication. This enables excellent scalability at least up to 32 k cores in magnetohydrodynamic tests depending on the problem and hardware. In the version of dccrg presented here part of the mesh metadata is replicated between MPI processes reducing the scalability of adaptive mesh refinement (AMR) to between 200 and 600 processes. Dccrg is free software that anyone can use, study and modify and is available at [https://gitorious.org/dccrg]. Users are also kindly requested to cite this work when publishing results obtained with dccrg.