Enhancing Dynamic Mode Decomposition Workflow with In-Situ Visualization and Data Compression

TL;DR

This paper improves Dynamic Mode Decomposition workflows in large-scale simulations by integrating in-situ visualization and data compression, significantly reducing storage needs and enabling real-time dynamic reconstruction with high accuracy.

Contribution

It introduces strategies for snapshot compression and in-situ visualization integration into DMD, enhancing efficiency and enabling real-time analysis in large simulations.

Findings

Lossy compression reduces storage by nearly 50% with low error.

In-situ visualization enables efficient, real-time data reconstruction.

Streaming DMD effectively updates modes with new data during runtime.

Abstract

Modern computational science and engineering applications are being improved by the advances in scientific machine learning. Data-driven methods such as Dynamic Mode Decomposition (DMD) can extract coherent structures from spatio-temporal data generated from dynamical systems and infer different scenarios for said systems. The spatio-temporal data comes as snapshots containing spatial information for each time instant. In modern engineering applications, the generation of high-dimensional snapshots can be time and/or resource-demanding. In the present study, we consider two strategies for enhancing DMD workflow in large numerical simulations: (i) snapshots compression to relieve disk pressure; (ii) the use of in situ visualization images to reconstruct the dynamics (or part of) in runtime. We evaluate our approaches with two 3D fluid dynamics simulations and consider DMD to reconstruct the solutions. Results reveal that snapshot compression considerably reduces the required disk space. We have observed that lossy compression reduces storage by almost $50\%$ with low relative errors in the signal reconstructions and other quantities of interest. We also extend our analysis to data generated on-the-fly, using in-situ visualization tools to generate image files of our state vectors during runtime. On large simulations, the generation of snapshots may be slow enough to use batch algorithms for inference. Streaming DMD takes advantage of the incremental SVD algorithm and updates the modes with the arrival of each new snapshot. We use streaming DMD to reconstruct the dynamics from in-situ generated images. We show that this process is efficient, and the reconstructed dynamics are accurate.