Adaptation and Development of Super Schemes for Unconstrained Optimization Problems

TL;DR

This paper introduces new super-schemes for unconstrained optimization that achieve higher-order convergence with low computational cost, outperforming classical gradient methods in efficiency and stability.

Contribution

The paper develops novel multi-step iterative schemes with high convergence orders, simple implementation, and broad applicability to large-scale and ill-conditioned problems.

Findings

Methods achieve convergence orders two, four, and six.

Numerical experiments show significant performance improvements over classical methods.

Computational complexity remains comparable to existing gradient-based methods.

Abstract

In this paper, we propose a class of super-schemes for efficiently solving nonlinear unconstrained optimization problems. The proposed approach introduces two novel choices of step-size parameters, leading to efficient descent directions without requiring second-order information. We develop one-step, two-step, and three-step iterative schemes (denoted by SS1, SS2, and SS3) and establish that these methods achieve higher-order convergence of orders two, four, and six, respectively. Despite their high convergence rates, the computational complexity of the proposed methods remains comparable to existing gradient-based methods, with a cost of $\mathcal{O}(n^2)$ per iteration. The proposed methods are simple to implement and do not require complicated line-search procedures. Their effectiveness is demonstrated through extensive numerical experiments on a wide range of problems, including large-scale and ill-conditioned cases. The results show that the proposed methods significantly outperform classical methods, such as the Barzilai-Borwein method and other gradient-based approaches, in terms of iteration count and computational efficiency. Finally, the numerical results are consistent with the theoretical analysis, confirming the stability of the proposed schemes for test optimization problems.