Learning embeddings of non-linear PDEs: the Burgers' equation

TL;DR

This paper introduces a method to embed solutions of nonlinear PDEs, specifically Burgers' equation, into a low-dimensional space using neural networks and PCA, enabling efficient analysis and physical interpretation.

Contribution

It generalizes embeddings to Physics Informed Neural Networks and develops a multi-head approach with orthogonality constraints for robust, interpretable solution space representations.

Findings

Latent modes capture dominant dynamics of Burgers' equation

Orthogonality constraints improve robustness and interpretability

Few modes are sufficient to represent key features

Abstract

Embeddings provide low-dimensional representations that organize complex function spaces and support generalization. They provide a geometric representation that supports efficient retrieval, comparison, and generalization. In this work we generalize the concept to Physics Informed Neural Networks. We present a method to construct solution embedding spaces of nonlinear partial differential equations using a multi-head setup, and extract non-degenerate information from them using principal component analysis (PCA). We test this method by applying it to viscous Burgers' equation, which is solved simultaneously for a family of initial conditions and values of the viscosity. A shared network body learns a latent embedding of the solution space, while linear heads map this embedding to individual realizations. By enforcing orthogonality constraints on the heads, we obtain a principal-component decomposition of the latent space that is robust to training degeneracies and admits a direct physical interpretation. The obtained components for Burgers' equation exhibit rapid saturation, indicating that a small number of latent modes captures the dominant features of the dynamics.