Proximal Multitask Learning over Networks with Sparsity-inducing Coregularization

TL;DR

This paper introduces a distributed multitask learning algorithm that promotes cooperation among networked clusters using sparsity-inducing regularizers, with proven convergence and demonstrated effectiveness through simulations.

Contribution

It develops a novel diffusion forward-backward splitting algorithm with a closed-form proximal operator for sparsity regularization in multitask network learning.

Findings

Effective cooperation among clusters improves estimation accuracy.

The algorithm converges under specified step-size conditions.

Simulations confirm the approach's efficiency and robustness.

Abstract

In this work, we consider multitask learning problems where clusters of nodes are interested in estimating their own parameter vector. Cooperation among clusters is beneficial when the optimal models of adjacent clusters have a good number of similar entries. We propose a fully distributed algorithm for solving this problem. The approach relies on minimizing a global mean-square error criterion regularized by non-differentiable terms to promote cooperation among neighboring clusters. A general diffusion forward-backward splitting strategy is introduced. Then, it is specialized to the case of sparsity promoting regularizers. A closed-form expression for the proximal operator of a weighted sum of $\ell_1$-norms is derived to achieve higher efficiency. We also provide conditions on the step-sizes that ensure convergence of the algorithm in the mean and mean-square error sense. Simulations are conducted to illustrate the effectiveness of the strategy.