SDCA without Duality, Regularization, and Individual Convexity

TL;DR

This paper introduces variants of Stochastic Dual Coordinate Ascent that operate without explicit regularization or duality, achieving linear convergence even with non-convex individual losses, as long as the expected loss is strongly convex.

Contribution

It presents a novel SDCA framework that removes the need for regularization and duality, extending applicability to non-convex individual losses with strong convexity in expectation.

Findings

Achieves linear convergence rates under new conditions.

Operates without explicit regularization or duality.

Applicable to non-convex individual loss functions.

Abstract

Stochastic Dual Coordinate Ascent is a popular method for solving regularized loss minimization for the case of convex losses. We describe variants of SDCA that do not require explicit regularization and do not rely on duality. We prove linear convergence rates even if individual loss functions are non-convex, as long as the expected loss is strongly convex.