A Fast Proximal Gradient Algorithm for Decentralized Composite Optimization over Directed Networks

TL;DR

This paper introduces PG-ExtraPush, a fast decentralized algorithm for composite optimization over directed networks, capable of handling nonconvex problems with improved convergence and efficiency.

Contribution

It extends existing algorithms to handle composite objectives over directed networks, combining proximity operators and push-sum protocol for faster convergence.

Findings

PG-ExtraPush converges linearly under certain conditions.

The algorithm outperforms Subgradient-Push in speed.

Effective in nonconvex and convex scenarios.

Abstract

This paper proposes a fast decentralized algorithm for solving a consensus optimization problem defined in a directed networked multi-agent system, where the local objective functions have the smooth+nonsmooth composite form, and are possibly nonconvex. Examples of such problems include decentralized compressed sensing and constrained quadratic programming problems, as well as many decentralized regularization problems. We extend the existing algorithms PG-EXTRA and ExtraPush to a new algorithm PG-ExtraPush for composite consensus optimization over a directed network. This algorithm takes advantage of the proximity operator like in PG-EXTRA to deal with the nonsmooth term, and employs the push-sum protocol like in ExtraPush to tackle the bias introduced by the directed network. With a proper step size, we show that PG-ExtraPush converges to an optimal solution at a linear rate under some regular assumptions. We conduct a series of numerical experiments to show the effectiveness of the proposed algorithm. Specifically, with a proper step size, PG-ExtraPush performs linear rates in most of cases, even in some nonconvex cases, and is significantly faster than Subgradient-Push, even if the latter uses a hand-optimized step size. The established theoretical results are also verified by the numerical results.