A Globally and Quadratically Convergent Algorithm with Efficient Implementation for Unconstrained Optimization

TL;DR

This paper introduces a modified Newton algorithm for unconstrained optimization that combines the identity matrix and Hessian dynamically, ensuring global and quadratic convergence with efficient implementation and promising numerical results.

Contribution

A novel modified Newton algorithm with a convex combination of Hessian and identity matrix, proven to be globally and quadratically convergent, and validated through extensive numerical testing.

Findings

Outperforms some existing algorithms on benchmark problems

Ensures global and quadratic convergence

Demonstrates efficiency in numerical experiments

Abstract

In this paper, an efficient modified Newton type algorithm is proposed for nonlinear unconstrianed optimization problems. The modified Hessian is a convex combination of the identity matrix (for steepest descent algorithm) and the Hessian matrix (for Newton algorithm). The coefficients of the convex combination are dynamically chosen in every iteration. The algorithm is proved to be globally and quadratically convergent for (convex and nonconvex) nonlinear functions. Efficient implementation is described. Numerical test on widely used CUTE test problems is conducted for the new algorithm. The test results are compared with those obtained by MATLAB optimization toolbox function {\tt fminunc}. The test results are also compared with those obtained by some established and state-of-the-art algorithms, such as a limited memory BFGS, a descent and conjugate gradient algorithm, and a limited memory and descent conjugate gradient algorithm. The comparisons show that the new algorithm is promising.