Performance Analysis of $l_0$ Norm Constrained Recursive Least Squares Algorithm

TL;DR

This paper provides a comprehensive theoretical performance analysis of the $l_0$ norm constrained Recursive Least Squares algorithm, including steady state mean square deviation and dynamic behavior, validated by numerical simulations.

Contribution

It offers the first thorough theoretical analysis of the $l_0$ RLS algorithm, deriving expressions for steady state MSD and its evolution, with validation through simulations.

Findings

Derived steady state MSD expression for $l_0$ RLS.

Analyzed the evolution of instantaneous MSD using Taylor series expansion.

Simulation results closely match theoretical predictions across various parameters.

Abstract

Performance analysis of $l_0$ norm constrained Recursive least Squares (RLS) algorithm is attempted in this paper. Though the performance pretty attractive compared to its various alternatives, no thorough study of theoretical analysis has been performed. Like the popular $l_0$ Least Mean Squares (LMS) algorithm, in $l_0$ RLS, a $l_0$ norm penalty is added to provide zero tap attractions on the instantaneous filter taps. A thorough theoretical performance analysis has been conducted in this paper with white Gaussian input data under assumptions suitable for many practical scenarios. An expression for steady state MSD is derived and analyzed for variations of different sets of predefined variables. Also a Taylor series expansion based approximate linear evolution of the instantaneous MSD has been performed. Finally numerical simulations are carried out to corroborate the theoretical analysis and are shown to match well for a wide range of parameters.