# Stochastic Primal-Dual Algorithms with Faster Convergence than $O(1/\sqrt{T})$ for Problems without Bilinear Structure

## TL;DR

This paper introduces new stochastic primal-dual algorithms that achieve faster convergence rates than the traditional $O(1/\sqrt{T})$ for convex-concave problems without requiring bilinear structure, applicable to robust learning and AUC maximization.

## Contribution

The paper develops and analyzes stochastic primal-dual algorithms with a mixture of stochastic and deterministic updates, achieving improved convergence rates for non-bilinear convex-concave problems.

## Key findings

- Achieves $O(1/T)$ convergence rate under certain conditions.
- Applicable to problems with weak strong convexity and strong concavity.
- Effective in robust model learning and empirical AUC maximization.

## Abstract

Previous studies on stochastic primal-dual algorithms for solving min-max problems with faster convergence heavily rely on the bilinear structure of the problem, which restricts their applicability to a narrowed range of problems. The main contribution of this paper is the design and analysis of new stochastic primal-dual algorithms that use a mixture of stochastic gradient updates and a logarithmic number of deterministic dual updates for solving a family of convex-concave problems with no bilinear structure assumed. Faster convergence rates than $O(1/\sqrt{T})$ with $T$ being the number of stochastic gradient updates are established under some mild conditions of involved functions on the primal and the dual variable. For example, for a family of problems that enjoy a weak strong convexity in terms of the primal variable and has a strongly concave function of the dual variable, the convergence rate of the proposed algorithm is $O(1/T)$. We also investigate the effectiveness of the proposed algorithms for learning robust models and empirical AUC maximization.