A Multi-Agent Primal-Dual Strategy for Composite Optimization over Distributed Features

TL;DR

This paper introduces a novel decentralized primal-dual algorithm for multi-agent composite optimization, achieving linear convergence even with non-smooth coupling functions, applicable to distributed machine learning tasks.

Contribution

It presents the first linearly convergent decentralized method for multi-agent sharing problems with general convex coupling functions.

Findings

Proposed a proximal primal-dual algorithm with linear convergence.

Applicable to distributed machine learning and resource allocation.

Handles non-smooth convex coupling functions effectively.

Abstract

This work studies multi-agent sharing optimization problems with the objective function being the sum of smooth local functions plus a convex (possibly non-smooth) function coupling all agents. This scenario arises in many machine learning and engineering applications, such as regression over distributed features and resource allocation. We reformulate this problem into an equivalent saddle-point problem, which is amenable to decentralized solutions. We then propose a proximal primal-dual algorithm and establish its linear convergence to the optimal solution when the local functions are strongly-convex. To our knowledge, this is the first linearly convergent decentralized algorithm for multi-agent sharing problems with a general convex (possibly non-smooth) coupling function.