A Common Interface for Automatic Differentiation

TL;DR

This paper introduces DifferentiationInterface.jl, a Julia package that unifies multiple AD backends through a common interface, facilitating comparison, modularity, and advanced features like sparsity handling in scientific machine learning.

Contribution

It provides a unified frontend for various AD systems in Julia, enabling easy comparison and advanced features without user burden.

Findings

Supports a dozen AD backends in Julia

Enables efficient comparison and modular development

Allows sophisticated features like sparsity handling

Abstract

For scientific machine learning tasks with a lot of custom code, picking the right Automatic Differentiation (AD) system matters. Our Julia package DifferentiationInterface$.$jl provides a common frontend to a dozen AD backends, unlocking easy comparison and modular development. In particular, its built-in preparation mechanism leverages the strengths of each backend by amortizing one-time computations. This is key to enabling sophisticated features like sparsity handling without putting additional burdens on the user.