Smooth Primal-Dual Coordinate Descent Algorithms for Nonsmooth Convex Optimization

TL;DR

This paper introduces a novel randomized coordinate descent algorithm for nonsmooth convex optimization, combining smoothing, acceleration, homotopy, and non-uniform sampling to achieve the best-known convergence rates.

Contribution

It presents the first convergence guarantees for coordinate descent methods under broad structural assumptions, advancing theoretical understanding and practical efficiency.

Findings

Achieves the best-known convergence rates for coordinate descent methods.

Demonstrates superior performance through numerical experiments.

Provides theoretical analysis supporting the effectiveness of the proposed approach.

Abstract

We propose a new randomized coordinate descent method for a convex optimization template with broad applications. Our analysis relies on a novel combination of four ideas applied to the primal-dual gap function: smoothing, acceleration, homotopy, and coordinate descent with non-uniform sampling. As a result, our method features the first convergence rate guarantees among the coordinate descent methods, that are the best-known under a variety of common structure assumptions on the template. We provide numerical evidence to support the theoretical results with a comparison to state-of-the-art algorithms.