Three-point Step Size Gradient Method with Relaxed Generalized Armijo Step Size Rule

TL;DR

This paper introduces a new three-point gradient method with a relaxed Armijo step size rule for unconstrained optimization, demonstrating global convergence, strong convergence for pseudo-convex functions, and suitability for large-scale problems.

Contribution

It proposes a novel three-point step size gradient method based on recent iteration information and Taylor's theorem, with proven convergence properties and efficiency for large-scale optimization.

Findings

Method achieves global convergence under certain conditions.

Strong convergence results for pseudo-convex functions.

Numerical experiments confirm efficiency and robustness.

Abstract

Based on differences of points and differences of gradients over the most recent three iterations, together with the Taylor's theorem, two forms of the quasi-Newton equations at the recent iteration are constructed. By using the two forms of the quasi-Newton equation and the method of least squares, three-point step size gradient methods for solving unconstrained optimization problem are proposed. It is proved by using the relaxed generalized Armijo step size rule that the new method is of global convergence properties if the gradient function is uniformly continuous. Moreover, it is shown that, when the objective function is pseudo-convex (quasi-convex) function, the new method has strong convergence results. In addition, it is also shown under some suitable assumptions that the new method is of super-linear and linear convergence. Although multi-piont information is used, TBB has the feature of simplicity, low memory requirement and only first order information being used, the new method is very suitable for solving large-scale optimization problems. Numerical experiments are provided and the efficiency, robustness and analysis of TBB are confirmed.