Accelerated Distributed Projected Gradient Descent for Convex Optimization with Clique-wise Coupled Constraints

TL;DR

This paper introduces an accelerated distributed gradient algorithm with clique-based projections for convex optimization under coupled constraints, demonstrating improved convergence rates and effectiveness through theoretical analysis and experiments.

Contribution

It proposes a novel clique-based projection operator and an accelerated distributed gradient method with proven convergence properties for convex optimization with clique-wise constraints.

Findings

Convergence to optimal solutions under diminishing step sizes.

Achieves O(1/k) convergence rate with fixed step sizes.

Improves to O(1/k^2) convergence with Nesterov's acceleration.

Abstract

This paper addresses a distributed convex optimization problem with a class of coupled constraints, which arise in a multi-agent system composed of multiple communities modeled by cliques. First, we propose a fully distributed gradient-based algorithm with a novel operator inspired by the convex projection, called the clique-based projection. Next, we scrutinize the convergence properties for both diminishing and fixed step sizes. For diminishing ones, we show the convergence to an optimal solution under the assumptions of the smoothness of an objective function and the compactness of the constraint set. Additionally, when the objective function is strongly monotone, the strict convergence to the unique solution is proved without the assumption of compactness. For fixed step sizes, we prove the non-ergodic convergence rate of O(1/k) concerning the objective residual under the assumption of the smoothness of the objective function. Furthermore, we apply Nesterov's acceleration method to the proposed algorithm and establish the convergence rate of O(1/k^2). Numerical experiments illustrate the effectiveness of the proposed method.