Stochastic Projective Splitting: Solving Saddle-Point Problems with Multiple Regularizers

TL;DR

This paper introduces a stochastic variant of projective splitting algorithms capable of efficiently solving saddle-point problems with multiple regularizers, addressing convergence issues in min-max and game formulations in machine learning.

Contribution

It is the first stochastic projective splitting method that handles multiple regularizers and constraints in saddle-point problems, improving robustness over gradient descent-ascent.

Findings

Successfully applied to distributionally robust sparse logistic regression

Handles multiple constraints and nonsmooth regularizers efficiently

Demonstrates improved convergence properties over existing methods

Abstract

We present a new, stochastic variant of the projective splitting (PS) family of algorithms for monotone inclusion problems. It can solve min-max and noncooperative game formulations arising in applications such as robust ML without the convergence issues associated with gradient descent-ascent, the current de facto standard approach in such situations. Our proposal is the first version of PS able to use stochastic (as opposed to deterministic) gradient oracles. It is also the first stochastic method that can solve min-max games while easily handling multiple constraints and nonsmooth regularizers via projection and proximal operators. We close with numerical experiments on a distributionally robust sparse logistic regression problem.