Benchmarking sparse system identification with low-dimensional chaos

TL;DR

This paper benchmarks various sparse system identification methods on chaotic systems, revealing the strengths of the original SINDy algorithm, the benefits of ensembling, and the robustness of weak SINDy formulations across different dynamical properties.

Contribution

It provides a comprehensive large-scale comparison of SINDy variants using the dysts chaotic systems database, highlighting the most effective algorithms and techniques.

Findings

Original SINDy performs strongly among methods.

Ensembling improves noise robustness of SINDy.

Weak SINDy significantly outperforms traditional methods.

Abstract

Sparse system identification is the data-driven process of obtaining parsimonious differential equations that describe the evolution of a dynamical system, balancing model complexity and accuracy. There has been rapid innovation in system identification across scientific domains, but there remains a gap in the literature for large-scale methodological comparisons that are evaluated on a variety of dynamical systems. In this work, we systematically benchmark sparse regression variants by utilizing the dysts standardized database of chaotic systems. In particular, we demonstrate how this open-source tool can be used to quantitatively compare different methods of system identification. To illustrate how this benchmark can be utilized, we perform a large comparison of four algorithms for solving the sparse identification of nonlinear dynamics (SINDy) optimization problem, finding strong performance of the original algorithm and a recent mixed-integer discrete algorithm. In all cases, we used ensembling to improve the noise robustness of SINDy and provide statistical comparisons. In addition, we show very compelling evidence that the weak SINDy formulation provides significant improvements over the traditional method, even on clean data. Lastly, we investigate how Pareto-optimal models generated from SINDy algorithms depend on the properties of the equations, finding that the performance shows no significant dependence on a set of dynamical properties that quantify the amount of chaos, scale separation, degree of nonlinearity, and the syntactic complexity.