A variation of Broyden Class methods using Householder adaptive transforms

TL;DR

This paper introduces a new class of Quasi Newton methods using adaptive Householder transforms for improved convergence and robustness in large-scale optimization problems.

Contribution

It proposes a novel Broyden Class-type updating scheme with adaptive Householder projections, enhancing convergence guarantees and robustness over traditional methods.

Findings

Adaptive methods improve robustness over non-adaptive schemes.

The proposed methods perform well on large-scale problems.

Theoretical convergence and quadratic termination are established.

Abstract

In this work we introduce and study novel Quasi Newton minimization methods based on a Hessian approximation Broyden Class-\textit{type} updating scheme, where a suitable matrix $\tilde{B}_k$ is updated instead of the current Hessian approximation $B_k$. We identify conditions which imply the convergence of the algorithm and, if exact line search is chosen, its quadratic termination. By a remarkable connection between the projection operation and Krylov spaces, such conditions can be ensured using low complexity matrices $\tilde{B}_k$ obtained projecting $B_k$ onto algebras of matrices diagonalized by products of two or three Householder matrices adaptively chosen step by step. Extended experimental tests show that the introduction of the adaptive criterion, which theoretically guarantees the convergence, considerably improves the robustness of the minimization schemes when compared with a non-adaptive choice; moreover, they show that the proposed methods could be particularly suitable to solve large scale problems where $L$-$BFGS$ performs poorly.