DQM: Decentralized Quadratically Approximated Alternating Direction Method of Multipliers

TL;DR

This paper introduces DQM, a decentralized optimization algorithm that reduces computational costs by using quadratic approximations, maintaining linear convergence similar to DADMM, and demonstrating improved efficiency in logistic regression tasks.

Contribution

The paper proposes DQM, a novel decentralized optimization method that approximates the objective quadratically to reduce computational complexity while preserving convergence rates.

Findings

DQM achieves similar linear convergence rates as DADMM.

DQM reduces computational time per iteration.

Numerical results show DQM outperforms DADMM in logistic regression.

Abstract

This paper considers decentralized consensus optimization problems where nodes of a network have access to different summands of a global objective function. Nodes cooperate to minimize the global objective by exchanging information with neighbors only. A decentralized version of the alternating directions method of multipliers (DADMM) is a common method for solving this category of problems. DADMM exhibits linear convergence rate to the optimal objective but its implementation requires solving a convex optimization problem at each iteration. This can be computationally costly and may result in large overall convergence times. The decentralized quadratically approximated ADMM algorithm (DQM), which minimizes a quadratic approximation of the objective function that DADMM minimizes at each iteration, is proposed here. The consequent reduction in computational time is shown to have minimal effect on convergence properties. Convergence still proceeds at a linear rate with a guaranteed constant that is asymptotically equivalent to the DADMM linear convergence rate constant. Numerical results demonstrate advantages of DQM relative to DADMM and other alternatives in a logistic regression problem.