Stochastic Primal Dual Coordinate Method with Non-Uniform Sampling Based on Optimality Violations

TL;DR

This paper introduces a convergence analysis for a stochastic primal-dual coordinate method with arbitrary sampling and proposes a new non-uniform sampling technique based on optimality violations, demonstrating improved efficiency over existing methods.

Contribution

The paper provides the first convergence analysis for primal-dual stochastic methods with arbitrary sampling and introduces a novel optimality violation-based sampling scheme.

Findings

The proposed ovsSPDC method outperforms state-of-the-art stochastic optimization algorithms in speed.

The heuristic variants ovsSDPC+ and ovsSDPC++ further improve efficiency.

Numerical experiments validate the theoretical advantages of the new sampling approach.

Abstract

We study primal-dual type stochastic optimization algorithms with non-uniform sampling. Our main theoretical contribution in this paper is to present a convergence analysis of Stochastic Primal Dual Coordinate (SPDC) Method with arbitrary sampling. Based on this theoretical framework, we propose Optimality Violation-based Sampling SPDC (ovsSPDC), a non-uniform sampling method based on Optimality Violation. We also propose two efficient heuristic variants of ovsSPDC called ovsSDPC+ and ovsSDPC++. Through intensive numerical experiments, we demonstrate that the proposed method and its variants are faster than other state-of-the-art primal-dual type stochastic optimization methods.

Full text

See pages - of paper-arxiv.pdf