Algorithmic Differentiation for Domain Specific Languages

TL;DR

This paper introduces a novel operator overloading approach for Algorithmic Differentiation (AD) tailored for Domain Specific Languages (DSLs), enabling optimized derivatives and improved performance by integrating DSL types into the AD process.

Contribution

The paper presents a new method that embeds DSL types into AD tools, allowing for compiler optimizations like SIMD and reducing memory usage, which improves derivative computation efficiency.

Findings

Enhanced performance demonstrated through initial results

Ability to optimize for SIMD operations within DSLs

Reduced memory requirements for AD processes

Abstract

Algorithmic Differentiation (AD) can be used to automate the generation of derivatives in arbitrary software projects. This will generate maintainable derivatives, that are always consistent with the computation of the software. If a domain specific language (DSL) is used in a software the state of the art approach is to differentiate the DSL library with the same AD tool. The drawback of this solution is the reduced performance since the compiler is no longer able to optimize the e.g. SIMD operations. The new approach in this paper integrates the types and operations of the DSL into the AD tool. It will be an operator overloading tool that is generated from an abstract definition of a DSL. This approach enables the compiler to optimize again e.g. for SIMD operation since all calculations are still performed with the original data types. This will also reduce the required memory for AD since the statements inside the DLS implementation are no longer seen by the AD tool. The implementation is presented in the paper and first results for the performance of the solution are presented.