Scalable CFD Simulations in Multi-Billion Voxel Micro-CT Images of Porous Materials Using OpenFOAM on ARCHER2

TL;DR

This paper demonstrates the use of high-performance computing with OpenFOAM on ARCHER2 to perform scalable, full-resolution CFD simulations on multi-billion voxel micro-CT images of porous rocks, avoiding complex meshing.

Contribution

It introduces a scalable CFD approach using volumetric indicator functions and Cartesian meshes for ultra-large micro-CT images, enabling detailed simulations without image size reduction.

Findings

Full-resolution simulations are feasible on large micro-CT images.

The Cartesian grid approach simplifies meshing and enhances scalability.

The method effectively handles heterogeneous porous structures.

Abstract

This study investigates the use of High-Performance Computing (HPC) to simulate flow and transport in ultra-large micro-CT images of porous materials using Computational Fluid Dynamics (CFD). Two distinct rock samples, representative of two different rock formations - Bentheimer sandstone and Estaillades carbonate - are investigated. The Bentheimer sandstone image, with dimensions 1,950x1,950x10,800 voxels at 6 micron resolution, comprising 41 billion voxels, represents a largely homogeneous structure, while the Estaillades carbonate image, at 1,144x1,144x6,000 voxels and 3.9676 micron resolution, amounting to 8 billion voxels, features greater heterogeneity, including micro-porous regions. These images are used for direct CFD simulations with GeoChemFoam, our OpenFOAM-based numerical solver, leveraging the computational resources of the UK supercomputer ARCHER2. One of the key aspects of the study is the use of the Darcy-Brinkman-Stokes approach, for which the solid surface is represented using a volumetric indicator function, rather than a complex mesh. This enables the use of simple Cartesian meshes that can be generated in parallel in an efficient and scalable manner. The study explores both weak and strong scaling through subvolume decomposition, demonstrating that, due to the strong scalability and the computational power of ARCHER2, full-resolution CFD simulations can be carried out without the need for image size reduction. This work illustrates the potential of HPC to perform detailed, full-scale simulations on large, high-resolution micro-CT data. The approach relies on a meshing strategy that leverages simple, parallelisable Cartesian grids derived from volumetric indicator functions, eliminating the need for complex surface-conforming meshes and allowing scalable simulation of flow and transport in geological and engineering applications.