Stochastic Primal-Dual Coordinate Method for Nonlinear Convex Cone Programs

TL;DR

This paper proposes a stochastic primal-dual coordinate method for large-scale nonlinear convex cone programs, offering convergence guarantees and complexity bounds, addressing limitations of existing block coordinate methods.

Contribution

It introduces a novel stochastic primal-dual coordinate algorithm specifically designed for large-scale NCCP problems, with proven convergence and complexity analysis.

Findings

Almost sure convergence to optimal solution

Two types of convergence rates established

Probability complexity bounds derived

Abstract

Block coordinate descent (BCD) methods and their variants have been widely used in coping with large-scale nonconstrained optimization problems in many fields such as imaging processing, machine learning, compress sensing and so on. For problem with coupling constraints, Nonlinear convex cone programs (NCCP) are important problems with many practical applications, but these problems are hard to solve by using existing block coordinate type methods. This paper introduces a stochastic primal-dual coordinate (SPDC) method for solving large-scale NCCP. In this method, we randomly choose a block of variables based on the uniform distribution. The linearization and Bregman-like function (core function) to that randomly selected block allow us to get simple parallel primal-dual decomposition for NCCP. The sequence generated by our algorithm is proved almost surely converge to an optimal solution of primal problem. Two types of convergence rate with different probability (almost surely and expected) are also obtained. The probability complexity bound is also derived in this paper.