Adding complex numbers to expression template algorithmic differentiation tools

TL;DR

This paper extends operator overloading algorithmic differentiation tools to natively support complex numbers, reducing memory use and improving computation speed by integrating complex operations directly into the AD framework.

Contribution

It introduces a method to incorporate complex number operations into modern expression template AD tools, enhancing efficiency and simplifying internal computations.

Findings

Reduced memory footprint in AD computations.

Faster gradient calculation times with complex number support.

Successful implementation and performance demonstration in CoDiPack.

Abstract

Operator overloading algorithmic differentiation (AD) tools are usually only developed for floating-point values. Algorithmic optimization for, e.g., linear systems solvers or matrix-matrix multiplications are often introduced via external functions or manual function specializations. Complex numbers can be viewed as aggregates of two floating-point values on which specialized operations are applied. Typically, these operations can be handled by the regular floating-point operations from the AD tool. Nevertheless, adding the complex number operations to the expression template framework of modern operator overloading AD tools has several benefits. The internal computations of a complex number operation are hidden, and the complex operations do not decompose into single operations. This leads to a smaller memory footprint of the recorded tape and faster gradient computation times. We will discuss these problems, analyze how complex numbers can be integrated into modern operator overloading AD tools, demonstrate an implementation in CoDiPack, and show performance results on a synthetic test case.