Sparse LMS via Online Linearized Bregman Iteration

TL;DR

This paper introduces OLBI, an online sparse LMS algorithm based on linearized Bregman iteration, which effectively identifies sparse systems with improved performance and theoretical convergence guarantees.

Contribution

The paper presents OLBI, a novel online sparse LMS algorithm derived from Bregman iteration, with bias-free properties and comprehensive convergence analysis.

Findings

OLBI is bias free and converges under certain conditions.

OLBI outperforms existing sparse LMS algorithms in simulations.

Theoretical expressions for steady state and instantaneous MSD are derived.

Abstract

We propose a version of least-mean-square (LMS) algorithm for sparse system identification. Our algorithm called online linearized Bregman iteration (OLBI) is derived from minimizing the cumulative prediction error squared along with an l1-l2 norm regularizer. By systematically treating the non-differentiable regularizer we arrive at a simple two-step iteration. We demonstrate that OLBI is bias free and compare its operation with existing sparse LMS algorithms by rederiving them in the online convex optimization framework. We perform convergence analysis of OLBI for white input signals and derive theoretical expressions for both the steady state and instantaneous mean square deviations (MSD). We demonstrate numerically that OLBI improves the performance of LMS type algorithms for signals generated from sparse tap weights.