New gradient methods with 3 dimensional quadratic termination

TL;DR

This paper introduces a novel gradient method with an innovative stepsize that, when combined with exact line search, efficiently solves quadratic minimization problems and extends to general unconstrained problems with proven convergence properties.

Contribution

The paper proposes a new stepsize for gradient methods, integrates it with cyclic update strategies, and extends the approach to general unconstrained problems with convergence analysis.

Findings

Achieves optimal solution in 5 steps for 3D quadratic problems.

Extends to general unconstrained problems using quadratic interpolation.

Demonstrates good computational performance with few line search trials.

Abstract

A new stepsize for gradient method is proposed. Combining it with the exact line search stepsizes, the gradient method achieves the optimal solution in 5 steps for 3 dimensional quadratic function minimization problem. The new stepsize is plugged in the cyclic stepsize update strategy, and a new gradient method is proposed. By applying the quadratic interpolation for Cauchy approximation, the proposed gradient method is extended to solve general unconstrained problem. With the improved GLL line search, the global convergence of the proposed method is proved. Furthermore, its sublinear convergence rate for convex problems and R-linear convergence rate for problems with quadratic functional growth property are analyzed. Numerical results show that our proposed algorithm enjoys good performances in terms of computational cost, and line search requires very few trial stepsizes.