On interpretability and proper latent decomposition of autoencoders

TL;DR

This paper provides a geometric interpretation of autoencoders' latent spaces in turbulence modeling, deriving the metric tensor and proposing a proper latent decomposition to identify dominant flow directions.

Contribution

It introduces a Riemannian geometric framework for analyzing autoencoder latent spaces and proposes a new decomposition method for turbulence data.

Findings

Latent space is a curved Riemannian manifold.

Derived the metric tensor for the autoencoder latent space.

Proposed proper latent decomposition (PLD) for turbulence analysis.

Abstract

The dynamics of a turbulent flow tend to occupy only a portion of the phase space at a statistically stationary regime. From a dynamical systems point of view, this portion is the attractor. The knowledge of the turbulent attractor is useful for two purposes, at least: (i) We can gain physical insight into turbulence (what is the shape and geometry of the attractor?), and (ii) it provides the minimal number of degrees of freedom to accurately describe the turbulent dynamics. Autoencoders enable the computation of an optimal latent space, which is a low-order representation of the dynamics. If properly trained and correctly designed, autoencoders can learn an approximation of the turbulent attractor, as shown by Doan, Racca and Magri (2022). In this paper, we theoretically interpret the transformations of an autoencoder. First, we remark that the latent space is a curved manifold with curvilinear coordinates, which can be analyzed with simple tools from Riemann geometry. Second, we characterize the geometrical properties of the latent space. We mathematically derive the metric tensor, which provides a mathematical description of the manifold. Third, we propose a method -- proper latent decomposition (PLD) -- that generalizes proper orthogonal decomposition of turbulent flows on the autoencoder latent space. This decomposition finds the dominant directions in the curved latent space. This theoretical work opens up computational opportunities for interpreting autoencoders and creating reduced-order models of turbulent flows.