Likelihood-based generalization of Markov parameter estimation and multiple shooting objectives in system identification

TL;DR

This paper introduces a Bayesian-derived objective function for system identification that unifies and improves upon existing methods like least squares and multiple shooting, especially under noisy or sparse data conditions.

Contribution

It proposes a novel Bayesian-based optimization framework that generalizes and enhances traditional system identification techniques, providing better regularization and robustness.

Findings

Lower mean squared error compared to multiple shooting in noisy/sparse data scenarios

Effective in identifying models with more parameters than data

Handles chaotic system behaviors accurately

Abstract

This paper considers the problem of system identification (ID) of linear and nonlinear non-autonomous systems from noisy and sparse data. We propose and analyze an objective function derived from a Bayesian formulation for learning a hidden Markov model with stochastic dynamics. We then analyze this objective function in the context of several state-of-the-art approaches for both linear and nonlinear system ID. In the former, we analyze least squares approaches for Markov parameter estimation, and in the latter, we analyze the multiple shooting approach. We demonstrate the limitations of the optimization problems posed by these existing methods by showing that they can be seen as special cases of the proposed optimization objective under certain simplifying assumptions: conditional independence of data and zero model error. Furthermore, we observe that our proposed approach has improved smoothness and inherent regularization that make it well-suited for system ID and provide mathematical explanations for these characteristics' origins. Finally, numerical simulations demonstrate a mean squared error over 8.7 times lower compared to multiple shooting when data are noisy and/or sparse. Moreover, the proposed approach can identify accurate and generalizable models even when there are more parameters than data or when the underlying system exhibits chaotic behavior.