Hessian Chain Bracketing

TL;DR

This paper introduces a dynamic programming approach to efficiently compute Hessians in numerical models, significantly reducing computational effort compared to existing methods.

Contribution

It formulates Hessian accumulation as an NP-complete problem and proposes a dynamic programming solution that outperforms current techniques.

Findings

Achieves tenfold or greater reduction in computation time

Provides a new formulation for Hessian accumulation as an NP-complete problem

Outperforms second-order tangent and adjoint algorithmic differentiation methods

Abstract

Second derivatives of mathematical models for real-world phenomena are fundamental ingredients of a wide range of numerical simulation methods including parameter sensitivity analysis, uncertainty quantification, nonlinear optimization and model calibration. The evaluation of such Hessians often dominates the overall computational effort. The combinatorial {\sc Hessian Accumulation} problem aiming to minimize the number of floating-point operations required for the computation of a Hessian turns out to be NP-complete. We propose a dynamic programming formulation for the solution of {\sc Hessian Accumulation} over a sub-search space. This approach yields improvements by factors of ten and higher over the state of the art based on second-order tangent and adjoint algorithmic differentiation.