The Golden Ratio Primal-Dual Algorithm with Two New Stepsize Rules for Convex-Concave Saddle Point Problems

TL;DR

This paper introduces two adaptive stepsize strategies for the Golden Ratio primal-dual algorithm to efficiently solve structured convex optimization problems, achieving sublinear and linear convergence rates without requiring Lipschitz constants.

Contribution

The paper proposes two novel adaptive stepsize rules for the extended Golden Ratio primal-dual algorithm, eliminating the need for Lipschitz constant computation and improving convergence guarantees.

Findings

Both stepsize rules achieve ergodic sublinear convergence.

The first stepsize rule attains R-linear convergence under standard assumptions.

Numerical experiments show improved performance over existing methods.

Abstract

In this paper, we present two stepsize strategies for the extended Golden Ratio primal-dual algorithm (E-GRPDA) designed to address structured convex optimization problems in finite-dimensional real Hilbert spaces. The first rule features a non-increasing primal stepsize that remains bounded below by a positive constant and is updated adaptively at each iteration, eliminating the need to compute the Lipschitz constant of the gradient of the function and the norm of the operator, without using backtracking. The second stepsize rule is adaptive, adjusting based on the local smoothness of the smooth component function and the norm of the operator involved. In other words, we present an adaptive version of the E-GRPDA algorithm. We prove that E-GRPDA achieves an ergodic sublinear convergence rate with both stepsize rules, based on the function-value residual and constraint violation rather than on the so-called primal-dual gap function. Additionally, we establish an R-linear convergence rate for E-GRPDA with the first stepsize rule, under standard assumptions and with appropriately chosen parameters. Through numerical experiments on various convex optimization problems, we demonstrate the effectiveness of our approaches and compare their performance to existing ones.