A New Complexity Result for Strongly Convex Optimization with Locally $\alpha$-H{\"o}lder Continuous Gradients

TL;DR

This paper establishes a new complexity bound for gradient descent on strongly convex functions with locally Hölder continuous gradients, extending previous results for the case when the gradient is Lipschitz continuous.

Contribution

It introduces a novel complexity analysis for gradient descent with fixed stepsize on functions with locally α-Hölder continuous gradients, generalizing known bounds.

Findings

Complexity bound of O(log(ε^{-1}) ε^{2α - 2}) for approximate minimizers

Extension of classical results from Lipschitz continuous gradients to Hölder continuous case

Provides theoretical insights into gradient descent performance under weaker smoothness conditions

Abstract

In this paper, we present a new complexity result for the gradient descent method with an appropriately fixed stepsize for minimizing a strongly convex function with locally $\alpha$-H{\"o}lder continuous gradients ($0 < \alpha \leq 1$). The complexity bound for finding an approximate minimizer with a distance to the true minimizer less than $\varepsilon$ is $O(\log (\varepsilon^{-1}) \varepsilon^{2 \alpha - 2})$, which extends the well-known complexity result for $\alpha = 1$.