$\beta$-Variational autoencoders and transformers for reduced-order modelling of fluid flows

TL;DR

This paper introduces a novel reduced-order modeling approach combining $eta$-VAE and transformers to efficiently capture and predict complex fluid flow dynamics, outperforming existing models in accuracy and interpretability.

Contribution

The paper presents a new method integrating $eta$-VAE and transformers for learning compact, interpretable, and dynamic latent representations of fluid flows, improving prediction accuracy.

Findings

The method captures flow dynamics more accurately than previous models.

It produces interpretable latent features resembling POD modes.

The approach outperforms other models in flow prediction tasks.

Abstract

Variational autoencoder (VAE) architectures have the potential to develop reduced-order models (ROMs) for chaotic fluid flows. We propose a method for learning compact and near-orthogonal ROMs using a combination of a $\beta$-VAE and a transformer, tested on numerical data from a two-dimensional viscous flow in both periodic and chaotic regimes. The $\beta$-VAE is trained to learn a compact latent representation of the flow velocity, and the transformer is trained to predict the temporal dynamics in latent space. Using the $\beta$-VAE to learn disentangled representations in latent-space, we obtain a more interpretable flow model with features that resemble those observed in the proper orthogonal decomposition, but with a more efficient representation. Using Poincar\'e maps, the results show that our method can capture the underlying dynamics of the flow outperforming other prediction models. The proposed method has potential applications in other fields such as weather forecasting, structural dynamics or biomedical engineering.