Decentralized Communication-Efficient Multi-Task Representation Learning

TL;DR

This paper introduces a decentralized, communication-efficient algorithm for multi-task representation learning that accurately recovers low-rank matrices using alternating projected gradient descent, even with non-convex constraints.

Contribution

It presents the first provably correct decentralized algorithm applicable to non-convex projection sets and for problems involving alternating projected gradient descent.

Findings

Algorithm achieves accurate low-rank matrix recovery

Ensures communication efficiency in decentralized settings

Works with non-convex constraint sets

Abstract

This work develops a provably accurate fully-decentralized alternating projected gradient descent (GD) algorithm for recovering a low rank (LR) matrix from mutually independent projections of each of its columns, in a fast and communication-efficient fashion. To our best knowledge, this work is the first attempt to develop a provably correct decentralized algorithm (i) for any problem involving the use of an alternating projected GD algorithm; (ii) and for any problem in which the constraint set to be projected to is a non-convex set.