Asymptotic Efficiency Analysis of the Recursive Least-Squares Algorithm for ARX Systems Without Projection

TL;DR

This paper proves that the recursive least-squares algorithm for ARX systems is asymptotically efficient and achieves the Cramér-Rao lower bound without requiring parameter boundedness assumptions, using a new analytical framework.

Contribution

It introduces a novel analysis method that removes the need for boundedness assumptions, establishing the RLS algorithm's asymptotic efficiency for ARX systems.

Findings

RLS achieves asymptotic normality under bounded twentieth-order moments.

The covariance matrix of RLS converges to the Cramér-Rao lower bound.

The new framework eliminates the need for projection operators in analysis.

Abstract

This paper investigates the optimality analysis of the recursive least-squares (RLS) algorithm for autoregressive systems with exogenous inputs (ARX systems). A key challenge in analyzing is managing the potential unboundedness of the parameter estimates, which may diverge to infinity. Previous approaches addressed this issue by assuming that both the true parameter and the RLS estimates remain confined within a known compact set, thereby ensuring uniform boundedness throughout the analysis. In contrast, we propose a new analytical framework that eliminates the need for such a boundness assumption. Specifically, we establish a quantitative relationship between the bounded moment conditions of quasi-stationary input/output signals and the convergence rate of the tail probability of the RLS estimation error. Based on this technique, we prove that when system inputs/outputs have bounded twentieth-order moments, the RLS algorithm achieves asymptotic normality and the covariance matrix of the RLS algorithm converges to the Cram\'er-Rao lower bound (CRLB), confirming its asymptotic efficiency. These results demonstrate that the RLS algorithm is an asymptotically optimal identification algorithm for ARX systems, even without the projection operators to ensure that parameter estimates reside within a prior known compact set.