Scalable Peaceman-Rachford Splitting Method with Proximal Terms

TL;DR

This paper introduces a stochastic version of the Peaceman-Rachford splitting method, called SS-PRSM, which achieves an $O(1/K)$ convergence rate, outperforming many existing stochastic algorithms in large-scale optimization.

Contribution

The paper proposes the first stochastic PRSM algorithm with proven convergence rate and demonstrates its superior performance and flexibility over existing methods.

Findings

SS-PRSM achieves $O(1/K)$ convergence rate.

Outperforms state-of-the-art stochastic algorithms.

Has low memory cost for large-scale problems.

Abstract

Along with developing of Peaceman-Rachford Splittling Method (PRSM), many batch algorithms based on it have been studied very deeply. But almost no algorithm focused on the performance of stochastic version of PRSM. In this paper, we propose a new stochastic algorithm based on PRSM, prove its convergence rate in ergodic sense, and test its performance on both artificial and real data. We show that our proposed algorithm, Stochastic Scalable PRSM (SS-PRSM), enjoys the $O(1/K)$ convergence rate, which is the same as those newest stochastic algorithms that based on ADMM but faster than general Stochastic ADMM (which is $O(1/\sqrt{K})$). Our algorithm also owns wide flexibility, outperforms many state-of-the-art stochastic algorithms coming from ADMM, and has low memory cost in large-scale splitting optimization problems.