An Accelerated Primal Dual Algorithm with Backtracking for Decentralized Constrained Optimization

TL;DR

This paper introduces a novel distributed accelerated primal-dual algorithm with backtracking for multi-agent constrained optimization, achieving optimal convergence without prior knowledge of smoothness constants, and demonstrating superior performance in numerical tests.

Contribution

The paper presents the first distributed primal-dual method with backtracking that adapts to unknown smoothness constants for constrained convex optimization.

Findings

Achieves $ ext{O}(1/K)$ convergence rate for sub-optimality, infeasibility, and consensus violation.

Automatically adapts to unknown Lipschitz constants using distributed backtracking.

Demonstrates improved performance on distributed QCQP and SVM training problems.

Abstract

We propose a distributed accelerated primal-dual method with backtracking (D-APDB) for cooperative multi-agent constrained consensus optimization problems over an undirected network of agents, where only those agents connected by an edge can directly communicate to exchange large-volume data vectors using a high-speed, short-range communication protocol, e.g., WiFi, and we also assume that the network allows for one-hop simple information exchange beyond immediate neighbors as in LoRaWAN protocol. The objective is to minimize the sum of agent-specific composite convex functions over agent-specific private constraint sets. Unlike existing decentralized primal-dual methods that require knowledge of the Lipschitz constants, D-APDB automatically adapts to unknown smoothness constants by employing a distributed backtracking step-size search. Each agent relies only on first-order oracles associated with its own objective and constraint functions and on local communications with the neighboring agents, without any prior knowledge of Lipschitz constants. We establish $\mathcal{O}(1/K)$ convergence guarantees for sub-optimality, infeasibility and consensus violation, under standard assumptions on smoothness and on the connectivity of the communication graph. To our knowledge, when nodes have private constraints, especially when they are nonlinear convex constraints onto which projections are not cheap to compute, D-APDB is the first distributed method with backtracking that achieves the optimal convergence rate for the class of constrained composite convex optimization problems. We also provide numerical results for D-APDB on a distributed QCQP problem and distributed primal SVM training to illustrate the potential performance gains that can be achieved by D-APDB.