Stochastic Primal Dual Hybrid Gradient Algorithm with Adaptive Step-Sizes

TL;DR

This paper introduces an adaptive stochastic primal-dual hybrid gradient algorithm with proven convergence, addressing the challenge of selecting step-sizes in large-scale convex optimization, and demonstrates its effectiveness in computed tomography.

Contribution

It proposes a new class of adaptive SPDHG algorithms with convergence guarantees and practical parameter-updating strategies.

Findings

Convergence of the proposed adaptive algorithms is theoretically proven.

Numerical experiments show improved performance in computed tomography.

The method effectively adapts step-sizes for large-scale convex problems.

Abstract

In this work we propose a new primal-dual algorithm with adaptive step-sizes. The stochastic primal-dual hybrid gradient (SPDHG) algorithm with constant step-sizes has become widely applied in large-scale convex optimization across many scientific fields due to its scalability. While the product of the primal and dual step-sizes is subject to an upper-bound in order to ensure convergence, the selection of the ratio of the step-sizes is critical in applications. Up-to-now there is no systematic and successful way of selecting the primal and dual step-sizes for SPDHG. In this work, we propose a general class of adaptive SPDHG (A-SPDHG) algorithms, and prove their convergence under weak assumptions. We also propose concrete parameters-updating strategies which satisfy the assumptions of our theory and thereby lead to convergent algorithms. Numerical examples on computed tomography demonstrate the effectiveness of the proposed schemes.