A Fully Adaptive Steepest Descent Method

TL;DR

This paper introduces a fully adaptive steepest descent method for pseudo-convex optimization that guarantees global convergence with a sublinear rate, without requiring parameter tuning.

Contribution

The paper proposes a novel adaptive steepest descent algorithm with deterministic step-size rules, achieving the best known convergence rate for non-convex objectives without parameter estimation.

Findings

Sequence of function values is strictly monotonic.

Method converges globally to the objective's optimum.

Preliminary tests show low computational costs.

Abstract

For solving pseudo-convex global optimization problems, we present a novel fully adaptive steepest descent method (or ASDM) without any hard-to-estimate parameters. For the step-size regulation in an $\varepsilon$-normalized direction, we use the deterministic rules, which were proposed in J. Optim. Theory Appl. (2019,\, DOI: 10.1007/S10957-019-01585-W). We obtained the optimistic convergence estimates for the generated by ASDM sequence of iteration points. Namely, the sequence of function values of iterates has the advantage of the strict monotonic behaviour and globally converges to the objective function optimum with the sublinear rate. This rate of convergence is now known to be the best for the steepest descent method in the non-convex objectives context. Preliminary computational tests confirm the efficiency of the proposed method and low computational costs for its realization.