Distributed Optimisation with Linear Equality and Inequality Constraints using PDMM

TL;DR

This paper extends the primal-dual method of multipliers (PDMM) to handle both linear equality and inequality constraints in distributed convex optimization, providing a faster alternative to existing methods like ADMM.

Contribution

It introduces a modified PDMM algorithm that incorporates inequality constraints without slack variables, using convex analysis and fixed-point theory, with proven convergence for various update schemes.

Findings

PDMM converges faster than extended ADMM in experiments.

The modified PDMM handles inequality constraints without slack variables.

Convergence is proven for synchronous, asynchronous, and lossy transmission schemes.

Abstract

In this paper, we consider the problem of distributed optimisation of a separable convex cost function over a graph, where every edge and node in the graph could carry both linear equality and/or inequality constraints. We show how to modify the primal-dual method of multipliers (PDMM), originally designed for linear equality constraints, such that it can handle inequality constraints as well. The proposed algorithm does not need any slack variables, which is similar to the recent work [1] which extends the alternating direction method of multipliers (ADMM) for addressing decomposable optimisation with linear equality and inequality constraints. Using convex analysis, monotone operator theory and fixed-point theory, we show how to derive the update equations of the modified PDMM algorithm by applying Peaceman-Rachford splitting to the monotonic inclusion related to the lifted dual problem. To incorporate the inequality constraints, we impose a non-negativity constraint on the associated dual variables. This additional constraint results in the introduction of a reflection operator to model the data exchange in the network, instead of a permutation operator as derived for equality constraint PDMM. Convergence for both synchronous and stochastic update schemes of PDMM are provided. The latter includes asynchronous update schemes and update schemes with transmission losses. Experiments show that PDMM converges notably faster than extended ADMM of [1].