(Adaptive) Scaled gradient methods beyond locally Holder smoothness: Lyapunov analysis, convergence rate and complexity

TL;DR

This paper analyzes scaled gradient methods for smooth convex functions with locally Holder continuous gradients, establishing convergence rates and complexity bounds, including an adaptive variant that automatically adjusts parameters for improved performance.

Contribution

It extends convergence analysis of scaled gradient methods to locally Holder smooth functions and introduces an adaptive algorithm with automatic parameter tuning.

Findings

SGA achieves global convergence under local smoothness.

Linear convergence is established under local strong convexity and KL inequality.

AdaSGA automatically adjusts parameters and attains global convergence with local linear rates.

Abstract

This paper addresses the unconstrained minimization of smooth convex functions whose gradients are locally Holder continuous. Building on these results, we analyze the Scaled Gradient Algorithm (SGA) under local smoothness assumptions, proving its global convergence and iteration complexity. Furthermore, under local strong convexity and the Kurdyka-Lojasiewicz (KL) inequality, we establish linear convergence rates and provide explicit complexity bounds. In particular, we show that when the gradient is locally Lipschitz continuous, SGA attains linear convergence for any KL exponent. We then introduce and analyze an adaptive variant of SGA (AdaSGA), which automatically adjusts the scaling and step-size parameters. For this method, we show global convergence, and derive local linear rates under strong convexity.