Sparse Distributed Learning via Heterogeneous Diffusion Adaptive Networks

TL;DR

This paper proposes a heterogeneous diffusion LMS strategy for sparse distributed estimation that selectively applies regularization to reduce computational costs while maintaining optimal performance.

Contribution

It introduces a novel approach where only some nodes use convex regularization, balancing performance and computational efficiency.

Findings

Selective regularization achieves the same performance as full regularization.

Less computational cost due to partial regularization.

Provides guidelines for choosing regularized nodes and optimal parameters.

Abstract

In-network distributed estimation of sparse parameter vectors via diffusion LMS strategies has been studied and investigated in recent years. In all the existing works, some convex regularization approach has been used at each node of the network in order to achieve an overall network performance superior to that of the simple diffusion LMS, albeit at the cost of increased computational overhead. In this paper, we provide analytical as well as experimental results which show that the convex regularization can be selectively applied only to some chosen nodes keeping rest of the nodes sparsity agnostic, while still enjoying the same optimum behavior as can be realized by deploying the convex regularization at all the nodes. Due to the incorporation of unregularized learning at a subset of nodes, less computational cost is needed in the proposed approach. We also provide a guideline for selection of the sparsity aware nodes and a closed form expression for the optimum regularization parameter.