PythonFOAM: In-situ data analyses with OpenFOAM and Python

TL;DR

PythonFOAM enables in-situ data analysis for OpenFOAM using Python, allowing real-time processing, distributed computations, and integration of machine learning tools directly within CFD simulations.

Contribution

The paper introduces a Python-based wrapper for OpenFOAM that facilitates in-situ data analysis, including distributed SVD and neural network deployment, enhancing real-time and scalable CFD data processing.

Findings

Distributed SVD scales effectively on multiple nodes.

In-situ Python analysis integrates seamlessly with OpenFOAM.

Deep neural networks can compress flow-field data in real-time.

Abstract

We outline the development of a general-purpose Python-based data analysis tool for OpenFOAM. Our implementation relies on the construction of OpenFOAM applications that have bindings to data analysis libraries in Python. Double precision data in OpenFOAM is cast to a NumPy array using the NumPy C-API and Python modules may then be used for arbitrary data analysis and manipulation on flow-field information. We highlight how the proposed wrapper may be used for an in-situ online singular value decomposition (SVD) implemented in Python and accessed from the OpenFOAM solver PimpleFOAM. Here, `in-situ' refers to a programming paradigm that allows for a concurrent computation of the data analysis on the same computational resources utilized for the partial differential equation solver. In addition, to demonstrate data-parallel analyses, we deploy a distributed SVD, which collects snapshot data across the ranks of a distributed simulation to compute the global left singular vectors. Crucially, both OpenFOAM and Python share the same message passing interface (MPI) communicator for this deployment which allows Python objects and functions to exchange NumPy arrays across ranks. Subsequently, we provide scaling assessments of this distributed SVD on multiple nodes of Intel Broadwell and KNL architectures for canonical test cases such as the large eddy simulations of a backward facing step and a channel flow at friction Reynolds number of 395. Finally, we demonstrate the deployment of a deep neural network for compressing the flow-field information using an autoencoder to demonstrate an ability to use state-of-the-art machine learning tools in the Python ecosystem.