Large-Scale Simulations of Fully Resolved Complex Moving Geometries with Partially Saturated Cells

TL;DR

This paper introduces an efficient method for simulating complex moving geometries in fluid flow using the Partially Saturated Cells Method integrated into the waLBerla framework, enabling large-scale parallel computations on CPU and GPU systems.

Contribution

It extends the lattice Boltzmann method with a novel mapping technique for complex geometries and demonstrates high performance and scalability on supercomputers.

Findings

Achieves up to 86% of peak performance on GPUs.

Maintains moderate overhead for solid-fluid coupling.

Successfully simulates a 4.3 billion cell scenario with excellent parallel efficiency.

Abstract

We employ the Partially Saturated Cells Method (PSM) to model the interaction between the fluid flow and solid moving objects as an extension to the conventional lattice Boltzmann method. We introduce an efficient and accurate method for mapping complex moving geometries onto uniform Cartesian grids suitable for massively parallel processing. A validation of the physical accuracy of the solid-fluid coupling and the proposed mapping of complex geometries ispresented. The implementation is integrated into the code generation pipeline of the waLBerla framework so that highly optimized kernels for CPU and GPU architectures become available. We study the node-level performance of the automatically generated solver routines. 71% of the peak performance can be achieved on CPU nodes and 86% on GPU accelerated nodes. Only a moderate overhead is observed for the processing of the solid-fluid coupling when compared to the fluids simulations without moving objects. Finally, a counter-rotating rotor is presented as a prototype industrial scenario, resulting in a mesh size involving up to 4.3 billion fluid grid cells. For this scenario, excellent parallel efficiency is reported in a strong scaling study on up to 32,768 CPU cores on the LUMI-C supercomputer and on up to 1,024 NVIDIA A100 GPUs on the JUWELS Booster system.