Automatic Differentiation: a look through Tensor and Operational Calculus

TL;DR

This paper explores automatic differentiation using tensor and operational calculus, providing a simple implementation and explanations suitable for those familiar with these mathematical frameworks.

Contribution

It introduces a tensor and operational calculus perspective to automatic differentiation with a clear implementation example.

Findings

Provides a tensor calculus-based explanation of automatic differentiation

Offers a simple implementation illustrating the calculus steps

Serves as supplementary learning material for advanced calculus learners

Abstract

In this paper we take a look at Automatic Differentiation through the eyes of Tensor and Operational Calculus. This work is best consumed as supplementary material for learning tensor and operational calculus by those already familiar with automatic differentiation. To that purpose, we provide a simple implementation of automatic differentiation, where the steps taken are explained in the language tensor and operational calculus.