Regularized parametric system identification: a decision-theoretic formulation

TL;DR

This paper introduces a decision-theoretic framework for parametric system identification that combines classical and regularized methods, improving robustness in small-sample and overparameterized scenarios.

Contribution

It develops a novel decision-theoretic formulation that unifies classical and regularized system identification approaches, enhancing model robustness.

Findings

The approach is robust to small sample sizes.

It effectively handles overparameterization.

Provides a bridge between classical and Bayesian methods.

Abstract

Parametric prediction error methods constitute a classical approach to the identification of linear dynamic systems with excellent large-sample properties. A more recent regularized approach, inspired by machine learning and Bayesian methods, has also gained attention. Methods based on this approach estimate the system impulse response with excellent small-sample properties. In several applications, however, it is desirable to obtain a compact representation of the system in the form of a parametric model. By viewing the identification of such models as a decision, we develop a decision-theoretic formulation of the parametric system identification problem that bridges the gap between the classical and regularized approaches above. Using the output-error model class as an illustration, we show that this decision-theoretic approach leads to a regularized method that is robust to small sample-sizes as well as overparameterization.