An accelerated proximal PRS-SQP algorithm with dual ascent-descent procedures for smooth composite optimization

TL;DR

This paper introduces an accelerated proximal PRS-SQP algorithm with dual ascent-descent procedures and hybrid acceleration, improving convergence and stability in smooth composite optimization.

Contribution

It generalizes dual update strategies, proposes a dual ascent-descent procedure, and incorporates hybrid acceleration to enhance convergence in composite optimization.

Findings

The algorithm achieves faster convergence in numerical experiments.

It maintains stability across various dual-update scenarios.

Theoretical convergence rates are established within the Kurdyka-Lojasiewicz framework.

Abstract

Conventional wisdom in composite optimization suggests augmented Lagrangian dual ascent (ALDA) in Peaceman-Rachford splitting (PRS) methods for dual feasibility. However, ALDA may fail when the primal iterate is a local minimum, a stationary point, or a coordinatewise solution of the highly nonconvex augmented Lagrangian function. Splitting sequential quadratic programming (SQP) methods utilize augmented Lagrangian dual descent (ALDD) to directly minimize the primal residual, circumventing the limitations of ALDA and achieving faster convergence in smooth optimization. This paper aims to present a fairly accessible generalization of two contrasting dual updates, ALDA and ALDD, for smooth composite optimization. A key feature of our PRS-SQP algorithm is its dual ascent-descent procedure, which provides a free direction rule for the dual updates and a new insight to explain the counterintuitive convergence behavior. Furthermore, we incorporate a hybrid acceleration technique that combines inertial extrapolation and back substitution to improve convergence. Theoretically, we establish the feasibility for a wider range of acceleration factors than previously known and derive convergence rates within the Kurdyka- Lojasiewicz framework. Numerical experiments validate the effectiveness and stability of the proposed method in various dual-update scenarios.