Do Proportionate Algorithms Exploit Sparsity?

TL;DR

This paper critically examines proportionate-type adaptive algorithms, revealing their limitations and poor performance in certain sparse, non-stationary, and compressible system scenarios through theoretical analysis and simulations.

Contribution

It provides the first theoretical analysis explaining the poor performance of proportionate algorithms in specific sparse and non-stationary contexts.

Findings

Proportionate algorithms perform poorly in certain sparse scenarios.

Theoretical justification for limitations of proportionate updates.

Simulation results support the theoretical findings.

Abstract

Adaptive filters exploiting sparsity have been a very active research field, among which the algorithms that follow the "proportional-update principle", the so-called proportionate-type algorithms, are very popular. Indeed, there are hundreds of works on proportionate-type algorithms and, therefore, their advantages are widely known. This paper addresses the unexplored drawbacks and limitations of using proportional updates and their practical impacts. Our findings include the theoretical justification for the poor performance of these algorithms in several sparse scenarios, and also when dealing with non-stationary and compressible systems. Simulation results corroborating the theory are presented.