Scaling techniques for $\epsilon$-subgradient projection methods

TL;DR

This paper introduces a variable metric and stepsize strategies for epsilon-subgradient projection methods to improve convergence in nonsmooth convex optimization, supported by theoretical analysis and practical experiments.

Contribution

It proposes a novel combination of variable metric and stepsize strategies for epsilon-subgradient methods, with convergence analysis and practical implementation insights.

Findings

Variable scaling improves convergence behavior.

Different stepsize strategies affect the speed of convergence.

Numerical experiments validate the effectiveness of the proposed methods.

Abstract

The recent literature on first order methods for smooth optimization shows that significant improvements on the practical convergence behaviour can be achieved with variable stepsize and scaling for the gradient, making this class of algorithms attractive for a variety of relevant applications. In this paper we introduce a variable metric in the context of the $\epsilon$-subgradient projection methods for nonsmooth, constrained, convex problems, in combination with two different stepsize selection strategies. We develop the theoretical convergence analysis of the proposed approach and we also discuss practical implementation issues, as the choice of the scaling matrix. In order to illustrate the effectiveness of the method, we consider a specific problem in the image restoration framework and we numerically evaluate the effects of a variable scaling and of the steplength selection strategy on the convergence behaviour.