Asymptotic Theory for Regularized System Identification Part I: Empirical Bayes Hyper-parameter Estimator

TL;DR

This paper develops an asymptotic distribution theory for the empirical Bayes hyper-parameter estimator in regularized system identification, revealing factors influencing convergence and validating results through simulations.

Contribution

It provides the first asymptotic distribution analysis for the EB hyper-parameter estimator in regularized system identification, highlighting key influencing factors.

Findings

Convergence in distribution of the EB hyper-parameter estimator is established.

Asymptotic distribution of the model estimator is derived.

Monte Carlo simulations confirm theoretical predictions.

Abstract

Regularized system identification is the major advance in system identification in the last decade. Although many promising results have been achieved, it is far from complete and there are still many key problems to be solved. One of them is the asymptotic theory, which is about convergence properties of the model estimators as the sample size goes to infinity. The existing related results for regularized system identification are about the almost sure convergence of various hyper-parameter estimators. A common problem of those results is that they do not contain information on the factors that affect the convergence properties of those hyper-parameter estimators, e.g., the regression matrix. In this paper, we tackle problems of this kind for the regularized finite impulse response model estimation with the empirical Bayes (EB) hyper-parameter estimator and filtered white noise input. In order to expose and find those factors, we study the convergence in distribution of the EB hyper-parameter estimator, and the asymptotic distribution of its corresponding model estimator. For illustration, we run Monte Carlo simulations to show the efficacy of our obtained theoretical results.