Highly Efficient Lattice-Boltzmann Multiphase Simulations of Immiscible Fluids at High-Density Ratios on CPUs and GPUs through Code Generation

TL;DR

This paper introduces a highly efficient, automatically generated lattice Boltzmann simulation code for multiphase flows with high-density ratios, optimized for CPUs and GPUs, enabling large-scale parallel simulations.

Contribution

It presents a novel, automated code generation approach for high-performance multiphase lattice Boltzmann simulations on CPUs and GPUs supporting high-density ratios.

Findings

Achieved near-perfect scaling on supercomputers with up to 2048 GPUs.

Demonstrated high efficiency and reduced memory footprint through kernel fusion.

Validated the simulation accuracy with a 3D rising air bubble scenario.

Abstract

A high-performance implementation of a multiphase lattice Boltzmann method based on the conservative Allen-Cahn model supporting high-density ratios and high Reynolds numbers is presented. Metaprogramming techniques are used to generate optimized code for CPUs and GPUs automatically. The coupled model is specified in a high-level symbolic description and optimized through automatic transformations. The memory footprint of the resulting algorithm is reduced through the fusion of compute kernels. A roofline analysis demonstrates the excellent efficiency of the generated code on a single GPU. The resulting single GPU code has been integrated into the multiphysics framework waLBerla to run massively parallel simulations on large domains. Communication hiding and GPUDirect-enabled MPI yield near-perfect scaling behaviour. Scaling experiments are conducted on the Piz Daint supercomputer with up to 2048 GPUs, simulating several hundred fully resolved bubbles. Further, validation of the implementation is shown in a physically relevant scenario-a three-dimensional rising air bubble in water.