AD in Fortran, Part 1: Design

TL;DR

This paper introduces language extensions to Fortran that embed automatic differentiation directly into the programming model, simplifying the use of AD in scientific computing tasks and making programs more concise.

Contribution

It proposes new Fortran language features for integrated forward and reverse AD, enabling easier and more abstract use of AD in scientific applications.

Findings

Programs become more concise and closer to mathematical notation.

Shared interfaces for optimization and root-finding tasks are enabled.

Extensions facilitate use of existing AD tools through code generation.

Abstract

We propose extensions to Fortran which integrate forward and reverse Automatic Differentiation (AD) directly into the programming model. Irrespective of implementation technology, embedding AD constructs directly into the language extends the reach and convenience of AD while allowing abstraction of concepts of interest to scientific-computing practice, such as root finding, optimization, and finding equilibria of continuous games. Multiple different subprograms for these tasks can share common interfaces, regardless of whether and how they use AD internally. A programmer can maximize a function F by calling a library maximizer, XSTAR=ARGMAX(F,X0), which internally constructs derivatives of F by AD, without having to learn how to use any particular AD tool. We illustrate the utility of these extensions by example: programs become much more concise and closer to traditional mathematical notation. A companion paper describes how these extensions can be implemented by a program that generates input to existing Fortran-based AD tools.