On the Equivalence of CoCoA+ and DisDCA

TL;DR

This paper proves that the CoCoA+ algorithm, under its recommended experimental setting, is mathematically equivalent to a practical variant of DisDCA, linking two distributed optimization methods.

Contribution

It establishes the theoretical equivalence between CoCoA+ and DisDCA under their typical experimental configurations.

Findings

CoCoA+ and DisDCA are equivalent in their practical implementations.

The equivalence clarifies the relationship between two popular distributed optimization algorithms.

This insight may influence future algorithm selection and development in distributed machine learning.

Abstract

In this document, we show that the algorithm CoCoA+ (Ma et al., ICML, 2015) under the setting used in their experiments, which is also the best setting suggested by the authors that proposed this algorithm, is equivalent to the practical variant of DisDCA (Yang, NIPS, 2013).