Distributed Strongly Convex Optimization
Konstantinos I. Tsianos, Michael G. Rabbat

TL;DR
This paper introduces a distributed algorithm for strongly convex constrained optimization that achieves near-optimal convergence rates in both online and batch settings, even with noisy subgradients.
Contribution
It presents a novel distributed optimization algorithm with proven convergence rates for strongly convex functions, applicable in online, batch, and noisy environments.
Findings
Achieves convergence rate of O(log (sqrt(n) T) / T) in distributed setting.
Works in both online and batch data scenarios.
Robust to additive zero-mean noise in subgradients.
Abstract
A lot of effort has been invested into characterizing the convergence rates of gradient based algorithms for non-linear convex optimization. Recently, motivated by large datasets and problems in machine learning, the interest has shifted towards distributed optimization. In this work we present a distributed algorithm for strongly convex constrained optimization. Each node in a network of n computers converges to the optimum of a strongly convex, L-Lipchitz continuous, separable objective at a rate O(log (sqrt(n) T) / T) where T is the number of iterations. This rate is achieved in the online setting where the data is revealed one at a time to the nodes, and in the batch setting where each node has access to its full local dataset from the start. The same convergence rate is achieved in expectation when the subgradients used at each node are corrupted with additive zero-mean noise.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
