Distributed Stochastic Block Coordinate Descent for Time-Varying Multi-Agent Optimization

TL;DR

This paper introduces a distributed stochastic block coordinate descent method for large-scale, time-varying multi-agent convex optimization, achieving optimal complexity bounds with convergence guarantees.

Contribution

It proposes a novel distributed block coordinate descent algorithm that operates on random blocks, improving efficiency and providing explicit convergence rates and complexity bounds.

Findings

Expected convergence rate of O(1/√T)

Explicit complexity bound of O(b^2/ε^2)

Achieves the best known dependency on ε and b in the literature

Abstract

In this paper, a class of large-scale distributed nonsmooth convex optimization problem over time-varying multi-agent network is investigated. Specifically, the decision space which can be split into several blocks of convex set is considered. We present a distributed block coordinate descent (DSBCD) method in which for each node, information communication with other agents and a block Bregman projection are performed in each iteration. In contrast to existing work, we do not require the projection is operated on the whole decision space. Instead, in each step, distributed projection procedure is performed on only one random block. The explicit formulation of the convergence level depending on random projection probabilities and network parameters is achieved. An expected $O(1/\sqrt{T})$ rate is achieved. In addition, we obtain an explicit $\mathcal{O}(b^2/\epsilon^2)$ complexity bound with target accuracy $\epsilon$ and characteristic constant factor $b$. The complexity with dependency on $\epsilon$ and $b$ is shown to be the best known in this literature.