Efficient Expression Templates for Operator Overloading-based Automatic Differentiation

TL;DR

This paper introduces optimized expression template techniques for operator overloading-based automatic differentiation in C++, significantly reducing duplicate computations and enhancing performance in large-scale applications.

Contribution

It presents new methods for improving expression template efficiency and implements them in the Sacado package for better automatic differentiation performance.

Findings

Enhanced efficiency demonstrated on test functions

Improved performance in large-scale fluid dynamics simulations

Reduced duplicate computation in expression trees

Abstract

Expression templates are a well-known set of techniques for improving the efficiency of operator overloading-based forward mode automatic differentiation schemes in the C++ programming language by translating the differentiation from individual operators to whole expressions. However standard expression template approaches result in a large amount of duplicate computation, particularly for large expression trees, degrading their performance. In this paper we describe several techniques for improving the efficiency of expression templates and their implementation in the automatic differentiation package Sacado. We demonstrate their improved efficiency through test functions as well as their application to differentiation of a large-scale fluid dynamics simulation code.