AD-HOC: A C++ Expression Template package for high-order derivatives backpropagation

TL;DR

AD-HOC is a C++ library enabling high-order derivatives computation efficiently within a single backpropagation pass, leveraging compile-time techniques and offering easy integration with existing AD tools.

Contribution

It introduces a header-only C++ package that computes arbitrary order derivatives in a single pass without source code generation, enhancing speed and usability.

Findings

Calculates derivatives of arbitrary order in a single backpropagation pass

Operates with speeds comparable to handwritten code

Integrates seamlessly with other AD tools

Abstract

This document presents a new C++ Automatic Differentiation (AD) tool, AD-HOC (Automatic Differentiation for High-Order Calculations). This tool aims to have the following features: -Calculation of user specified derivatives of arbitrary order -To be able to run with similar speeds as handwritten code -All derivatives calculations are computed in a single backpropagation tree pass -No source code generation is used, relying heavily on the C++ compiler to statically build the computation tree before runtime -A simple interface -The ability to be used \textit{in conjunction} with other established, general-purpose dynamic AD tools -Header-only library, with no external dependencies -Open source, with a business-friendly license