Fej\'er-monotone hybrid steepest descent method for affinely constrained and composite convex minimization tasks

TL;DR

This paper proposes the FM-HSDM algorithm, a new method for solving affinely constrained convex minimization problems in Hilbert spaces, offering convergence guarantees, flexibility, and efficiency for large-scale tasks.

Contribution

The paper introduces FM-HSDM, a novel Fejér-monotone hybrid steepest descent method that handles affine constraints more flexibly than existing primal-dual and ADMM methods.

Findings

Sequences converge weakly and strongly to solutions.

Algorithm accommodates affine constraints more flexibly.

Suitable for large-scale optimization with low computational cost.

Abstract

This paper introduces the Fej\'er-monotone hybrid steepest descent method (FM-HSDM), a new member to the HSDM family of algorithms, for solving affinely constrained minimization tasks in real Hilbert spaces, where convex smooth and non-smooth losses compose the objective function. FM-HSDM offers sequences of estimates which converge weakly and, under certain hypotheses, strongly to solutions of the task at hand. Fixed-point theory, variational inequalities and affine-nonexpansive mappings are utilized to devise a scheme that accommodates affine constraints in a more versatile way than state-of-the-art primal-dual techniques and the alternating direction method of multipliers do. Recursions can be tuned to score low computational footprints, well-suited for large-scale optimization tasks, without compromising convergence guarantees. In contrast to its HSDM's precursors, FM-HSDM enjoys Fej\'er monotonicity, the step-size parameter stays constant across iterations to promote convergence speed-ups of the sequence of estimates to a minimizer, while only Lipschitzian continuity, and not strong monotonicity, of the derivative of the smooth-loss function is needed to ensure convergence. Results on the rate of convergence to an optimal point are also presented.