Regularized Least-Mean-Square Algorithms

TL;DR

This paper introduces a family of regularized LMS algorithms for adaptive system identification, demonstrating improved mean square deviation performance and proposing methods for selecting regularization parameters, especially for sparse systems.

Contribution

It develops a new regularized LMS framework with provable dominance over traditional LMS and provides explicit formulas for optimal regularization parameter selection.

Findings

Regularized LMS algorithms outperform conventional LMS in mean square deviation.

Proposed sparse and group-sparse LMS algorithms excel in identifying sparse systems.

Simulation results confirm faster convergence and lower steady-state error.

Abstract

We consider adaptive system identification problems with convex constraints and propose a family of regularized Least-Mean-Square (LMS) algorithms. We show that with a properly selected regularization parameter the regularized LMS provably dominates its conventional counterpart in terms of mean square deviations. We establish simple and closed-form expressions for choosing this regularization parameter. For identifying an unknown sparse system we propose sparse and group-sparse LMS algorithms, which are special examples of the regularized LMS family. Simulation results demonstrate the advantages of the proposed filters in both convergence rate and steady-state error under sparsity assumptions on the true coefficient vector.