Distributed Mini-Batch SDCA

TL;DR

This paper provides an enhanced theoretical analysis of distributed mini-batch stochastic dual coordinate ascent algorithms, accommodating various sampling schemes and data distributions for efficient large-scale empirical loss minimization.

Contribution

It introduces a flexible analysis framework that accounts for data distribution and loss smoothness, improving understanding of distributed mini-batch SDCA performance.

Findings

Analysis depends on loss smoothness and data spectral norm

Supports distributed data sampling schemes

Improves convergence guarantees for large-scale optimization

Abstract

We present an improved analysis of mini-batched stochastic dual coordinate ascent for regularized empirical loss minimization (i.e. SVM and SVM-type objectives). Our analysis allows for flexible sampling schemes, including where data is distribute across machines, and combines a dependence on the smoothness of the loss and/or the data spread (measured through the spectral norm).