A new framework for the computation of Hessians

TL;DR

This paper introduces a novel framework combining graph and algebraic models for Hessian computation via Automatic Differentiation, leading to an efficient reverse algorithm that exploits symmetry and outperforms existing methods.

Contribution

It presents a new dual-model framework and a reverse Hessian algorithm, edge_pushing, that fully leverages Hessian symmetry for improved computational efficiency.

Findings

Edge_pushing outperforms existing algorithms on benchmark functions.

The framework reveals symmetries in Hessian calculations.

Experimental results show significant performance gains.

Abstract

We investigate the computation of Hessian matrices via Automatic Differentiation, using a graph model and an algebraic model. The graph model reveals the inherent symmetries involved in calculating the Hessian. The algebraic model, based on Griewank and Walther's state transformations, synthesizes the calculation of the Hessian as a formula. These dual points of view, graphical and algebraic, lead to a new framework for Hessian computation. This is illustrated by developing edge_pushing, a new truly reverse Hessian computation algorithm that fully exploits the Hessian's symmetry. Computational experiments compare the performance of edge_pushing on sixteen functions from the CUTE collection against two algorithms available as drivers of the software ADOL-C, and the results are very promising.