New convergence results for the scaled gradient projection method

TL;DR

This paper advances the theoretical understanding of the scaled gradient projection (SGP) method by establishing convergence to minimum points and an O(1/k) rate under convexity and Lipschitz conditions, supported by numerical experiments.

Contribution

It provides the first convergence proof for SGP to a minimum point under practical conditions and demonstrates an O(1/k) convergence rate for convex functions.

Findings

SGP converges to a minimum point under convexity and simple scaling conditions.

An O(1/k) convergence rate is established for the objective function values.

Numerical experiments show effective performance in image restoration tasks.

Abstract

The aim of this paper is to deepen the convergence analysis of the scaled gradient projection (SGP) method, proposed by Bonettini et al. in a recent paper for constrained smooth optimization. The main feature of SGP is the presence of a variable scaling matrix multiplying the gradient, which may change at each iteration. In the last few years, an extensive numerical experimentation showed that SGP equipped with a suitable choice of the scaling matrix is a very effective tool for solving large scale variational problems arising in image and signal processing. In spite of the very reliable numerical results observed, only a weak, though very general, convergence theorem is provided, establishing that any limit point of the sequence generated by SGP is stationary. Here, under the only assumption that the objective function is convex and that a solution exists, we prove that the sequence generated by SGP converges to a minimum point, if the scaling matrices sequence satisfies a simple and implementable condition. Moreover, assuming that the gradient of the objective function is Lipschitz continuous, we are also able to prove the O(1/k) convergence rate with respect to the objective function values. Finally, we present the results of a numerical experience on some relevant image restoration problems, showing that the proposed scaling matrix selection rule performs well also from the computational point of view.