Approximating the diagonal of a Hessian: which sample set of points should be used

TL;DR

This paper presents an explicit, efficient method for approximating the diagonal of a Hessian matrix using a single additional function evaluation, with error bounds guiding optimal sample set selection.

Contribution

It introduces a new explicit formula for Hessian diagonal approximation that requires only one extra function evaluation when combined with the generalized centered simplex gradient.

Findings

The formula provides an $ ext{O}( riangle_S^2)$ accuracy under specific sample set configurations.

An error bound guides the optimal construction of sample sets for Hessian diagonal approximation.

The method is computationally efficient and suitable for derivative-free optimization contexts.

Abstract

An explicit formula to approximate the diagonal entries of the Hessian is introduced. When the derivative-free technique called \emph{generalized centered simplex gradient} is used to approximate the gradient, then the formula can be computed for only one additional function evaluation. An error bound is introduced and provides information on the form of the sample set of points that should be used to approximate the diagonal of a Hessian. If the sample set of points is built in a specific manner, it is shown that the technique is $\mathcal{O}(\Delta_S^2)$ accurate approximation of the diagonal entries of the Hessian where $\Delta_S$ is the radius of the sample set.