Tangent-Space Regularization for Neural-Network Models of Dynamical Systems

TL;DR

This paper proposes tangent space regularization for neural networks modeling dynamical systems, improving learning efficiency and stability by leveraging properties of the system's Jacobian and tangent space.

Contribution

It introduces tangent space regularization for neural networks, enhancing their ability to learn dynamical systems with less data and better stability properties.

Findings

Regularization improves model accuracy on prediction and simulation tasks.

Different architectures vary in their ability to learn correct Jacobians.

Weight regularization influences system stability through Jacobian eigenvalues.

Abstract

This work introduces the concept of tangent space regularization for neural-network models of dynamical systems. The tangent space to the dynamics function of many physical systems of interest in control applications exhibits useful properties, e.g., smoothness, motivating regularization of the model Jacobian along system trajectories using assumptions on the tangent space of the dynamics. Without assumptions, large amounts of training data are required for a neural network to learn the full non-linear dynamics without overfitting. We compare different network architectures on one-step prediction and simulation performance and investigate the propensity of different architectures to learn models with correct input-output Jacobian. Furthermore, the influence of $L_2$ weight regularization on the learned Jacobian eigenvalue spectrum, and hence system stability, is investigated.