A variational derivation of a class of BFGS-like methods

TL;DR

This paper introduces a maximum entropy derivation for a new family of BFGS-like optimization methods, including block variants, providing theoretical insights and an independent proof of existing results.

Contribution

It presents a novel maximum entropy derivation for BFGS-like methods and extends the results to block BFGS methods, offering new theoretical understanding.

Findings

Derivation of a new family of BFGS-like methods using maximum entropy principles

Extension of results to block BFGS methods

Independent proof of Fletcher's 1991 result and its generalization

Abstract

We provide a maximum entropy derivation of a new family of BFGS-like methods. Similar results are then derived for block BFGS methods. This also yields an independent proof of a result of Fletcher 1991 and its generalisation to the block case.