Automatic Differentiation: Theory and Practice

TL;DR

This paper provides a formal, coordinate-free framework for automatic differentiation in both real and complex settings, deriving forward and backward mode formulas for matrix functions from fundamental principles.

Contribution

It introduces a classical, coordinate-free formalism for automatic differentiation, detailing derivations for matrix functions in real and complex contexts.

Findings

Formal derivation of forward and backward mode AD formulas

Coordinate-free approach applicable to real and complex matrices

Foundational framework for automatic differentiation methods

Abstract

We present the classical coordinate-free formalism for forward and backward mode ad in the real and complex setting. We show how to formally derive the forward and backward formulae for a number of matrix functions starting from basic principles.