H\"older Gradient Descent and Adaptive Regularization Methods in Banach Spaces for First-Order Points

TL;DR

This paper introduces a H"older gradient descent algorithm and analyzes the evaluation complexity of an adaptive regularization method for optimizing smooth nonconvex functionals in infinite-dimensional Banach spaces, focusing on first-order points.

Contribution

It proposes a novel H"older gradient descent method and provides complexity bounds for adaptive regularization in Banach spaces, extending optimization theory to infinite dimensions.

Findings

Evaluation complexity is at most O(ε^{-(p+β)/(p+β-1)}) for finding ε-approximate first-order points.

The methods are applicable to functionals with β-H"older continuous derivatives.

The approach extends finite-dimensional optimization techniques to infinite-dimensional Banach spaces.

Abstract

This paper considers optimization of smooth nonconvex functionals in smooth infinite dimensional spaces. A H\"older gradient descent algorithm is first proposed for finding approximate first-order points of regularized polynomial functionals. This method is then applied to analyze the evaluation complexity of an adaptive regularization method which searches for approximate first-order points of functionals with $\beta$-H\"older continuous derivatives. It is shown that finding an $\epsilon$-approximate first-order point requires at most $O(\epsilon^{-\frac{p+\beta}{p+\beta-1}})$ evaluations of the functional and its first $p$ derivatives.