Linear system identification using stable spline kernels and PLQ penalties

TL;DR

This paper introduces a flexible, regularized least squares approach for linear system identification using stable spline kernels and PLQ penalties, avoiding model order selection and handling outliers effectively.

Contribution

It develops new nonsmooth formulations for stable spline estimators, incorporating piecewise linear quadratic penalties and polyhedral constraints, solved efficiently with interior point methods.

Findings

Effective handling of outliers in measurements

Broad applicability to various regularization and data misfit functions

Efficient solution method with polynomial complexity per iteration

Abstract

The classical approach to linear system identification is given by parametric Prediction Error Methods (PEM). In this context, model complexity is often unknown so that a model order selection step is needed to suitably trade-off bias and variance. Recently, a different approach to linear system identification has been introduced, where model order determination is avoided by using a regularized least squares framework. In particular, the penalty term on the impulse response is defined by so called stable spline kernels. They embed information on regularity and BIBO stability, and depend on a small number of parameters which can be estimated from data. In this paper, we provide new nonsmooth formulations of the stable spline estimator. In particular, we consider linear system identification problems in a very broad context, where regularization functionals and data misfits can come from a rich set of piecewise linear quadratic functions. Moreover, our anal- ysis includes polyhedral inequality constraints on the unknown impulse response. For any formulation in this class, we show that interior point methods can be used to solve the system identification problem, with complexity O(n3)+O(mn2) in each iteration, where n and m are the number of impulse response coefficients and measurements, respectively. The usefulness of the framework is illustrated via a numerical experiment where output measurements are contaminated by outliers.