Fast-Slow Neural Networks for Learning Singularly Perturbed Dynamical Systems

TL;DR

This paper introduces a structure-preserving neural network approach for modeling singularly perturbed dynamical systems, effectively capturing slow manifolds and improving prediction accuracy in complex physical systems.

Contribution

It proposes the Fast-Slow Neural Network (FSNN) that enforces the existence of an attracting slow manifold, enabling efficient and accurate data-driven modeling of fast-slow systems.

Findings

Enforces a trainable slow manifold as a hard constraint.

Achieves improved prediction accuracy beyond training data.

Successfully applied to systems like Lorenz96 and Abraham-Lorentz dynamics.

Abstract

Singularly perturbed dynamical systems play a crucial role in climate dynamics and plasma physics. A powerful and well-known tool to address these systems is the Fenichel normal form, which significantly simplifies fast dynamics near slow manifolds through a transformation. However, this normal form is difficult to realize in conventional numerical algorithms. In this work, we explore an alternative way of realizing it through structure-preserving machine learning. Specifically, a fast-slow neural network (FSNN) is proposed for learning data-driven models of singularly perturbed dynamical systems with dissipative fast timescale dynamics. Our method enforces the existence of a trainable, attracting invariant slow manifold as a hard constraint. Closed-form representation of the slow manifold enables efficient integration on the slow time scale and significantly improves prediction accuracy beyond the training data. We demonstrate the FSNN on examples including the Grad moment system, two-scale Lorenz96 equations, and Abraham-Lorentz dynamics modeling radiation reaction of electrons.