Differentiable Scripting

TL;DR

This paper introduces a scripting language design that enforces differentiability in computational scripts, enabling derivative-based methods in scientific and financial applications, demonstrated through a prototype compiler.

Contribution

It proposes a novel scripting language approach that ensures differentiability by design, integrating differentiable subprograms and intrinsics, and avoiding nondifferentiable branches.

Findings

Prototype compiler demonstrates differentiability enforcement

Supports derivative-based error analysis and optimization

Enhances computational scripting for scientific applications

Abstract

In Computational Science, Engineering and Finance (CSEF) scripts typically serve as the "glue" between potentially highly complex and computationally expensive external subprograms. Differentiability of the resulting programs turns out to be essential in the context of derivative-based methods for error analysis, uncertainty quantification, optimization or training of surrogates. We argue that it should be enforced by the scripting language itself through exclusive support of differentiable (smoothed) external subprograms and differentiable intrinsics combined with prohibition of nondifferentiable branches in the data flow. Illustration is provided by a prototype adjoint code compiler for a simple Python-like scripting language.