# Accelerated Stochastic Algorithms for Convex-Concave Saddle-Point Problems

## TL;DR

This paper introduces new stochastic primal-dual algorithms with restart schemes that improve convergence rates for convex-concave saddle-point problems, especially under strong convexity and sub-Gaussian noise conditions.

## Contribution

The paper presents the first stochastic restart scheme for strongly convex saddle-point problems and a new primal-dual hybrid gradient-based algorithm with optimal oracle complexity.

## Key findings

- Restart scheme outperforms existing methods in oracle complexity.
- New algorithm achieves state-of-the-art oracle complexity for non-strongly convex cases.
- Method is nearly optimal under known lower bounds.

## Abstract

We develop stochastic first-order primal-dual algorithms to solve a class of convex-concave saddle-point problems. When the saddle function is strongly convex in the primal variable, we develop the first stochastic restart scheme for this problem. When the gradient noises obey sub-Gaussian distributions, the oracle complexity of our restart scheme is strictly better than any of the existing methods, even in the deterministic case. Furthermore, for each problem parameter of interest, whenever the lower bound exists, the oracle complexity of our restart scheme is either optimal or nearly optimal (up to a log factor). The subroutine used in this scheme is itself a new stochastic algorithm developed for the problem where the saddle function is non-strongly convex in the primal variable. This new algorithm, which is based on the primal-dual hybrid gradient framework, achieves the state-of-the-art oracle complexity and may be of independent interest.