Diffusion LMS for Multitask Problems with Local Linear Equality Constraints

TL;DR

This paper introduces a diffusion LMS algorithm for distributed multitask learning with linear equality constraints, enabling agents to optimize local costs while satisfying network-wide relationships, with theoretical analysis and practical applications.

Contribution

It proposes a novel adaptive stochastic algorithm based on projection gradient and diffusion strategies for constrained multitask learning in networks.

Findings

The algorithm converges with predictable mean-square-error behavior.

Closed-form expressions for learning dynamics are derived.

Simulations confirm the theoretical predictions and demonstrate practical utility.

Abstract

We consider distributed multitask learning problems over a network of agents where each agent is interested in estimating its own parameter vector, also called task, and where the tasks at neighboring agents are related according to a set of linear equality constraints. Each agent possesses its own convex cost function of its parameter vector and a set of linear equality constraints involving its own parameter vector and the parameter vectors of its neighboring agents. We propose an adaptive stochastic algorithm based on the projection gradient method and diffusion strategies in order to allow the network to optimize the individual costs subject to all constraints. Although the derivation is carried out for linear equality constraints, the technique can be applied to other forms of convex constraints. We conduct a detailed mean-square-error analysis of the proposed algorithm and derive closed-form expressions to predict its learning behavior. We provide simulations to illustrate the theoretical findings. Finally, the algorithm is employed for solving two problems in a distributed manner: a minimum-cost flow problem over a network and a space-time varying field reconstruction problem.