Sparse identification of evolution equations via Bayesian model selection

TL;DR

This paper presents a Bayesian optimization-based method for sparse identification of nonlinear dynamical systems from data, effectively handling challenges like derivative estimation and sampling rates, and demonstrating superior performance on real-world and simulated data.

Contribution

It introduces a novel approach combining thresholded least squares, error measures, and Bayesian optimization for hyperparameter tuning, with distinct regularization for each differential equation, advancing data-driven system identification.

Findings

Accurately models transient dynamics and oscillations in wake flow data.

Extracts meaningful differential equations from fMRI data, aiding explainable AI.

Outperforms existing methods in identifying sparse, accurate models.

Abstract

The quantitative formulation of evolution equations is the backbone for prediction, control, and understanding of dynamical systems across diverse scientific fields. Besides deriving differential equations for dynamical systems based on basic scientific reasoning or prior knowledge in recent times a growing interest emerged to infer these equations purely from data. In this article, we introduce a novel method for the sparse identification of nonlinear dynamical systems from observational data, based on the observation how the key challenges of the quality of time derivatives and sampling rates influence this problem. Our approach combines system identification based on thresholded least squares minimization with additional error measures that account for both the deviation between the model and the time derivative of the data, and the integrated performance of the model in forecasting dynamics. Specifically, we integrate a least squares error as well as the Wasserstein metric for estimated models and combine them within a Bayesian optimization framework to efficiently determine optimal hyperparameters for thresholding and weighting of the different error norms. Additionally, we employ distinct regularization parameters for each differential equation in the system, enhancing the method's precision and flexibility. We demonstrate the capabilities of our approach through applications to dynamical fMRI data and the prototypical example of a wake flow behind a cylinder. In the wake flow problem, our method identifies a sparse, accurate model that correctly captures transient dynamics, oscillation periods, and phase information, outperforming existing methods. In the fMRI example, we show how our approach extracts insights from a trained recurrent neural network, offering a novel avenue for explainable AI by inferring differential equations that capture potentially causal relationships.