Nearly Optimal Linear Convergence of Stochastic Primal-Dual Methods for Linear Programming

TL;DR

This paper introduces a stochastic primal-dual algorithm with variance reduction and restarts for linear programming, achieving nearly optimal linear convergence rates with low per-iteration complexity.

Contribution

The paper presents a novel stochastic method with variance reduction and restart techniques that attains nearly optimal linear convergence for primal-dual LP problems.

Findings

Achieves linear convergence with high probability on sharp LP instances.

Develops an efficient coordinate-based stochastic oracle for bilinear problems.

Improves complexity bounds of existing stochastic algorithms.

Abstract

There is a recent interest on first-order methods for linear programming (LP). In this paper,we propose a stochastic algorithm using variance reduction and restarts for solving sharp primal-dual problems such as LP. We show that the proposed stochastic method exhibits a linear convergence rate for solving sharp instances with a high probability. In addition, we propose an efficient coordinate-based stochastic oracle for unconstrained bilinear problems, which has $\mathcal O(1)$ per iteration cost and improves the complexity of the existing deterministic and stochastic algorithms. Finally, we show that the obtained linear convergence rate is nearly optimal (upto $\log$ terms) for a wide class of stochastic primal dual methods.