Unlocking massively parallel spectral proper orthogonal decompositions in the PySPOD package

TL;DR

This paper introduces a parallel spectral proper orthogonal decomposition (SPOD) algorithm implemented in the PySPOD library, enabling efficient analysis of large datasets in fluid dynamics and geophysics by leveraging distributed computing.

Contribution

The paper presents a novel parallel SPOD algorithm that preserves data structure and is implemented in an open-source Python library, facilitating large-scale data analysis.

Findings

Achieves scalable performance for large datasets

Enables analysis of previously intractable fluid dynamics data

Demonstrates effectiveness through fluid dynamics and geophysics applications

Abstract

We propose a parallel (distributed) version of the spectral proper orthogonal decomposition (SPOD) technique. The parallel SPOD algorithm distributes the spatial dimension of the dataset preserving time. This approach is adopted to preserve the non-distributed fast Fourier transform of the data in time, thereby avoiding the associated bottlenecks. The parallel SPOD algorithm is implemented in the PySPOD (https://github.com/MathEXLab/PySPOD) library and makes use of the standard message passing interface (MPI) library, implemented in Python via mpi4py (https://mpi4py.readthedocs.io/en/stable/). An extensive performance evaluation of the parallel package is provided, including strong and weak scalability analyses. The open-source library allows the analysis of large datasets of interest across the scientific community. Here, we present applications in fluid dynamics and geophysics, that are extremely difficult (if not impossible) to achieve without a parallel algorithm. This work opens the path toward modal analyses of big quasi-stationary data, helping to uncover new unexplored spatio-temporal patterns.