Modeling Nonlinear Dynamics in Continuous Time with Inductive Biases on Decay Rates and/or Frequencies

TL;DR

This paper introduces a neural network model for continuous-time nonlinear dynamics that incorporates inductive biases on decay rates and frequencies, improving forecasting especially with limited data.

Contribution

It leverages Koopman operator theory with neural networks to impose decay and frequency biases, enhancing modeling of nonlinear dynamics from small datasets.

Findings

Achieves higher forecasting accuracy with short training sequences.

Effectively incorporates decay rates and frequencies via Koopman eigenvalues.

Outperforms existing methods on various time-series datasets.

Abstract

We propose a neural network-based model for nonlinear dynamics in continuous time that can impose inductive biases on decay rates and/or frequencies. Inductive biases are helpful for training neural networks especially when training data are small. The proposed model is based on the Koopman operator theory, where the decay rate and frequency information is used by restricting the eigenvalues of the Koopman operator that describe linear evolution in a Koopman space. We use neural networks to find an appropriate Koopman space, which are trained by minimizing multi-step forecasting and backcasting errors using irregularly sampled time-series data. Experiments on various time-series datasets demonstrate that the proposed method achieves higher forecasting performance given a single short training sequence than the existing methods.