Quasi-Quadratic Gradient: A New Direction for Accelerating the BFGS Method in Quasi-Newton Optimization

TL;DR

This paper proposes the Quasi-Quadratic Gradient (QQG), a new search direction that accelerates BFGS optimization by leveraging local curvature, leading to faster convergence without extra computational cost.

Contribution

Introduction of QQG as a novel search direction that enhances BFGS convergence speed by explicitly utilizing local second-order curvature information.

Findings

QQG significantly outperforms vanilla BFGS in convergence speed.

The approach maintains computational efficiency.

Theoretical analysis supports the empirical improvements.

Abstract

In this paper, we introduce the Quasi-Quadratic Gradient (QQG), a novel search direction designed to accelerate the BFGS method within the quasi-Newton framework. By defining the QQG as the product of the inverse Hessian approximation and the current gradient, we explicitly leverage local second-order curvature to rectify the search path. Theoretical analysis and empirical results demonstrate that our approach significantly outperforms vanilla BFGS in convergence speed while maintaining computational efficiency.