Recent theoretical advances in decentralized distributed convex optimization

TL;DR

This paper reviews recent theoretical progress in decentralized distributed convex optimization, highlighting optimal algorithms and bounds, and introduces new unpublished results connecting distributed and non-distributed optimization methods.

Contribution

It explains recent advances by relating decentralized algorithms to optimal non-distributed algorithms and presents new unpublished results in the field.

Findings

Lower bounds on communication rounds established

Algorithms achieving these bounds identified

Connections between distributed and non-distributed optimization clarified

Abstract

In the last few years, the theory of decentralized distributed convex optimization has made significant progress. The lower bounds on communications rounds and oracle calls have appeared, as well as methods that reach both of these bounds. In this paper, we focus on how these results can be explained based on optimal algorithms for the non-distributed setup. In particular, we provide our recent results that have not been published yet and that could be found in details only in arXiv preprints.