On Geometry Regularization in Autoencoder Reduced-Order Models with Latent Neural ODE Dynamics

TL;DR

This paper explores various geometric regularization techniques for autoencoder-based reduced-order models with neural ODE dynamics, finding that Stiefel projection improves latent dynamics conditioning and long-term prediction accuracy.

Contribution

It systematically compares four regularization strategies in latent space for neural ODE-based reduced-order models, highlighting the effectiveness of Stiefel projection.

Findings

Stiefel projection improves latent dynamics conditioning

Regularization methods like Jacobian near-isometry can hinder long-term predictions

Latent-geometry mismatch impacts downstream model performance

Abstract

We investigate geometric regularization strategies for learned latent representations in encoder--decoder reduced-order models. In a fixed experimental setting for the advection--diffusion--reaction (ADR) equation, we model latent dynamics using a neural ODE and evaluate four regularization approaches applied during autoencoder pre-training: (a) near-isometry regularization of the decoder Jacobian, (b) a stochastic decoder gain penalty based on random directional gains, (c) a second-order directional curvature penalty, and (d) Stiefel projection of the first decoder layer. Across multiple seeds, we find that (a)--(c) often produce latent representations that make subsequent latent-dynamics training with a frozen autoencoder more difficult, especially for long-horizon rollouts, even when they improve local decoder smoothness or related sensitivity proxies. In contrast, (d) consistently improves conditioning-related diagnostics of the learned latent dynamics and tends to yield better rollout performance. We discuss the hypothesis that, in this setting, the downstream impact of latent-geometry mismatch outweighs the benefits of improved decoder smoothness.