A Weak Penalty Neural ODE for Learning Chaotic Dynamics from Noisy Time Series

TL;DR

This paper introduces a weak penalty approach for Neural ODEs that improves modeling of chaotic systems from noisy data, enhancing forecast accuracy and robustness.

Contribution

It proposes a weak formulation loss function for Neural ODEs, effectively filtering noise and overcoming instability in chaotic system modeling.

Findings

Weak penalty Neural ODEs outperform standard NODE in chaotic systems.

The approach is computationally efficient and solver-agnostic.

It yields accurate forecasts on benchmark and real-world climate data.

Abstract

The accurate forecasting of complex, high-dimensional dynamical systems from observational data is a fundamental task across numerous scientific and engineering disciplines. A significant challenge arises from noise-corrupted measurements, which severely degrade the performance of data-driven models. In chaotic dynamical systems, where small initial errors amplify exponentially, it is particularly difficult to develop a model from noisy data that achieves short-term accuracy while preserving long-term invariant properties. To overcome this, we consider the weak formulation as a complementary approach to the classical $L2$-loss function for training models of dynamical systems. We empirically verify that the weak formulation, with a proper choice of test function and integration domain, effectively filters noisy data. This insight explains why a weak form loss function is analogous to fitting a model to filtered data and provides a practical way to parameterize the weak form. Subsequently, we demonstrate how this approach overcomes the instability and inaccuracy of standard Neural ODE (NODE) in modeling chaotic systems. Through numerical examples, we show that our proposed training strategy, the Weak Penalty NODE, is computationally efficient, solver-agnostic, and yields accurate and robust forecasts across benchmark chaotic systems and a real-world climate dataset.