Robust EM kernel-based methods for linear system identification

TL;DR

This paper introduces a robust kernel-based system identification method that effectively handles outliers by modeling noise with heavy-tailed distributions and employing a novel EM-based hyperparameter estimation scheme.

Contribution

It develops a new robustification technique for kernel-based system identification using heavy-tailed noise models and a MAP estimator with an EM algorithm for hyperparameter tuning.

Findings

Significant performance improvement over existing methods in presence of outliers

Effective modeling of noise with Laplacian and Student's t distributions

Validated on simulated and real data

Abstract

Recent developments in system identification have brought attention to regularized kernel-based methods. This type of approach has been proven to compare favorably with classic parametric methods. However, current formulations are not robust with respect to outliers. In this paper, we introduce a novel method to robustify kernel-based system identification methods. To this end, we model the output measurement noise using random variables with heavy-tailed probability density functions (pdfs), focusing on the Laplacian and the Student's t distributions. Exploiting the representation of these pdfs as scale mixtures of Gaussians, we cast our system identification problem into a Gaussian process regression framework, which requires estimating a number of hyperparameters of the data size order. To overcome this difficulty, we design a new maximum a posteriori (MAP) estimator of the hyperparameters, and solve the related optimization problem with a novel iterative scheme based on the Expectation-Maximization (EM) method. In presence of outliers, tests on simulated data and on a real system show a substantial performance improvement compared to currently used kernel-based methods for linear system identification.