CoCoA: A General Framework for Communication-Efficient Distributed Optimization
Virginia Smith, Simone Forte, Chenxin Ma, Martin Takac, Michael I., Jordan, Martin Jaggi

TL;DR
CoCoA is a versatile framework for communication-efficient distributed optimization that effectively handles various regularizers and loss functions, demonstrating superior performance on real datasets.
Contribution
The paper introduces CoCoA, a general framework for distributed optimization with improved communication efficiency and convergence guarantees for a wide range of regularizers and loss functions.
Findings
Outperforms state-of-the-art methods in distributed settings
Handles non-strongly-convex regularizers like L1 and elastic net
Provides convergence guarantees for convex regularized objectives
Abstract
The scale of modern datasets necessitates the development of efficient distributed optimization methods for machine learning. We present a general-purpose framework for distributed computing environments, CoCoA, that has an efficient communication scheme and is applicable to a wide variety of problems in machine learning and signal processing. We extend the framework to cover general non-strongly-convex regularizers, including L1-regularized problems like lasso, sparse logistic regression, and elastic net regularization, and show how earlier work can be derived as a special case. We provide convergence guarantees for the class of convex regularized loss minimization objectives, leveraging a novel approach in handling non-strongly-convex regularizers and non-smooth loss functions. The resulting framework has markedly improved performance over state-of-the-art methods, as we illustrate…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsStochastic Gradient Optimization Techniques · Optimization and Search Problems · Complexity and Algorithms in Graphs
