Stochastic Parallel Block Coordinate Descent for Large-scale Saddle Point Problems

TL;DR

This paper introduces a stochastic parallel block coordinate descent method with adaptive primal-dual updates for large-scale convex-concave saddle point problems, enhancing efficiency and flexibility in machine learning applications.

Contribution

It proposes a novel stochastic block coordinate descent algorithm that leverages problem structure for improved parallel optimization in large-scale saddle point problems.

Findings

Outperforms existing methods in machine learning tasks

Effective for problems like robust PCA, Lasso, and group Lasso

Demonstrates both theoretical and empirical advantages

Abstract

We consider convex-concave saddle point problems with a separable structure and non-strongly convex functions. We propose an efficient stochastic block coordinate descent method using adaptive primal-dual updates, which enables flexible parallel optimization for large-scale problems. Our method shares the efficiency and flexibility of block coordinate descent methods with the simplicity of primal-dual methods and utilizing the structure of the separable convex-concave saddle point problem. It is capable of solving a wide range of machine learning applications, including robust principal component analysis, Lasso, and feature selection by group Lasso, etc. Theoretically and empirically, we demonstrate significantly better performance than state-of-the-art methods in all these applications.