Neural Ordinary Differential Equations for Nonlinear System Identification

TL;DR

Neural ordinary differential equations (NODE) significantly improve nonlinear system prediction accuracy over traditional methods, offering a promising approach with manageable computational costs and robustness to hyperparameters.

Contribution

This work provides a systematic comparison of NODEs with existing nonlinear and linear system identification methods, highlighting their superior prediction accuracy and robustness.

Findings

NODEs outperform classical and neural state-space models in prediction accuracy

NODEs are less sensitive to hyperparameter variations

Inference with NODEs incurs slightly higher computational cost

Abstract

Neural ordinary differential equations (NODE) have been recently proposed as a promising approach for nonlinear system identification tasks. In this work, we systematically compare their predictive performance with current state-of-the-art nonlinear and classical linear methods. In particular, we present a quantitative study comparing NODE's performance against neural state-space models and classical linear system identification methods. We evaluate the inference speed and prediction performance of each method on open-loop errors across eight different dynamical systems. The experiments show that NODEs can consistently improve the prediction accuracy by an order of magnitude compared to benchmark methods. Besides improved accuracy, we also observed that NODEs are less sensitive to hyperparameters compared to neural state-space models. On the other hand, these performance gains come with a slight increase of computation at the inference time.