Regularization-Induced Bias and Consistency in Recursive Least Squares

TL;DR

This paper investigates how regularization affects bias and accuracy in recursive least squares (RLS) for system identification, analyzing transient and asymptotic behaviors across different data and system types.

Contribution

It provides a comprehensive analysis of regularization-induced bias in RLS, relating condition numbers to convergence and accuracy in various system identification scenarios.

Findings

Regularization introduces bias affecting transient and asymptotic accuracy.

Condition number of regressors influences convergence rate.

Analysis covers random data, FIR, and IIR system identification.

Abstract

Within the context of recursive least squares (RLS) parameter estimation, the goal of the present paper is to study the effect of regularization-induced bias on the transient and asymptotic accuracy of the parameter estimates. We consider this question in three stages. First, we consider regression with random data, in which case persistency is guaranteed. Next, we apply RLS to finite-impulse-response (FIR) system identification and, finally, to infinite-impulse-response (IIR) system identification. For each case, we relate the condition number of the regressor matrix to the transient response and rate of convergence of the parameter estimates.