Decentralized Consensus Optimization Based on Parallel Random Walk

TL;DR

This paper introduces parallel and intelligent variants of random walk ADMM for decentralized consensus optimization, significantly improving communication efficiency and convergence speed over existing methods.

Contribution

It proposes PW-ADMM and IPW-ADMM algorithms that enhance communication efficiency and reduce running time in decentralized optimization.

Findings

PW-ADMM reduces communication costs compared to traditional ADMM.

IPW-ADMM accelerates convergence by integrating Random Walk with Choice.

Numerical results show the proposed methods outperform state-of-the-art algorithms.

Abstract

The alternating direction method of multipliers (ADMM) has recently been recognized as a promising approach for large-scale machine learning models. However, very few results study ADMM from the aspect of communication costs, especially jointly with running time. In this letter, we investigate the communication efficiency and running time of ADMM in solving the consensus optimization problem over decentralized networks. We first review the effort of random walk ADMM (W-ADMM), which reduces communication costs at the expense of running time. To accelerate the convergence speed of W-ADMM, we propose the parallel random walk ADMM (PW-ADMM) algorithm, where multiple random walks are active at the same time. Moreover, to further reduce the running time of PW-ADMM, the intelligent parallel random walk ADMM (IPW-ADMM) algorithm is proposed through integrating the \textit{Random Walk with Choice} with PW-ADMM. By numerical results from simulations, we demonstrate that the proposed algorithms can be both communication efficient and fast in running speed compared with state-of-the-art methods.