Greedy Sparsity-Promoting Algorithms for Distributed Learning

TL;DR

This paper introduces new greedy algorithms for distributed sparse learning, including batch and online versions, with theoretical convergence guarantees and improved performance demonstrated through simulations.

Contribution

The paper presents two novel greedy algorithms for distributed sparsity-aware learning, extending to online settings with theoretical convergence analysis.

Findings

Batch algorithm converges under RIP conditions.

Online algorithm converges in mean under general assumptions.

Proposed methods outperform recent sparsity-promoting algorithms in simulations.

Abstract

This paper focuses on the development of novel greedy techniques for distributed learning under sparsity constraints. Greedy techniques have widely been used in centralized systems due to their low computational requirements and at the same time their relatively good performance in estimating sparse parameter vectors/signals. The paper reports two new algorithms in the context of sparsity--aware learning. In both cases, the goal is first to identify the support set of the unknown signal and then to estimate the non--zero values restricted to the active support set. First, an iterative greedy multi--step procedure is developed, based on a neighborhood cooperation strategy, using batch processing on the observed data. Next, an extension of the algorithm to the online setting, based on the diffusion LMS rationale for adaptivity, is derived. Theoretical analysis of the algorithms is provided, where it is shown that the batch algorithm converges to the unknown vector if a Restricted Isometry Property (RIP) holds. Moreover, the online version converges in the mean to the solution vector under some general assumptions. Finally, the proposed schemes are tested against recently developed sparsity--promoting algorithms and their enhanced performance is verified via simulation examples.