Dealing with collinearity in large-scale linear system identification using Bayesian regularization

TL;DR

This paper introduces a Bayesian regularization approach with a novel MCMC scheme to identify large-scale, correlated linear systems efficiently, even under severe collinearity and ill-conditioning.

Contribution

It proposes a new Bayesian regularization method combined with an innovative MCMC scheme tailored for collinear, large-scale system identification.

Findings

Effective reconstruction of impulse responses in highly correlated systems

Robustness to severe collinearity and ill-conditioning

Successful application to systems with hundreds of responses

Abstract

We consider the identification of large-scale linear and stable dynamic systems whose outputs may be the result of many correlated inputs. Hence, severe ill-conditioning may affect the estimation problem. This is a scenario often arising when modeling complex physical systems given by the interconnection of many sub-units where feedback and algebraic loops can be encountered. We develop a strategy based on Bayesian regularization where any impulse response is modeled as the realization of a zero-mean Gaussian process. The stable spline covariance is used to include information on smooth exponential decay of the impulse responses. We then design a new Markov chain Monte Carlo scheme that deals with collinearity and is able to efficiently reconstruct the posterior of the impulse responses. It is based on a variation of Gibbs sampling which updates possibly overlapping blocks of the parameter space on the basis of the level of collinearity affecting the different inputs. Numerical experiments are included to test the goodness of the approach where hundreds of impulse responses form the system and inputs correlation may be very high.