Bayesian and regularization approaches to multivariable linear system identification: the role of rank penalties

TL;DR

This paper presents a Bayesian regularization method with a rank penalty for multivariable linear system identification, improving impulse response estimation by balancing smoothness, stability, and complexity.

Contribution

It introduces a novel impulse response estimator using a log-det heuristic for rank penalization within a Bayesian framework, enhancing model accuracy and interpretability.

Findings

Outperforms classic $ ext{l}_2$ regularization methods.

Superior to atomic and nuclear norm-based estimators.

Effectively enforces coupling in MIMO systems.

Abstract

Recent developments in linear system identification have proposed the use of non-parameteric methods, relying on regularization strategies, to handle the so-called bias/variance trade-off. This paper introduces an impulse response estimator which relies on an $\ell_2$-type regularization including a rank-penalty derived using the log-det heuristic as a smooth approximation to the rank function. This allows to account for different properties of the estimated impulse response (e.g. smoothness and stability) while also penalizing high-complexity models. This also allows to account and enforce coupling between different input-output channels in MIMO systems. According to the Bayesian paradigm, the parameters defining the relative weight of the two regularization terms as well as the structure of the rank penalty are estimated optimizing the marginal likelihood. Once these hyperameters have been estimated, the impulse response estimate is available in closed form. Experiments show that the proposed method is superior to the estimator relying on the "classic" $\ell_2$-regularization alone as well as those based in atomic and nuclear norm.