Sparse Automatic Differentiation for Complex Networks of Differential-Algebraic Equations Using Abstract Elementary Algebra

TL;DR

This paper proposes using abstract elementary algebra in numerical solvers to enhance flexibility and efficiency, demonstrated through sparse Jacobian matrix computation with scalable, parallelizable automatic differentiation.

Contribution

It introduces a novel approach to implement automatic differentiation using abstract algebra, enabling scalable and flexible solutions for complex differential-algebraic networks.

Findings

Sparse Jacobian computation benefits from abstract algebra implementation.

Automatic differentiation scales linearly with system size.

Parallelization in distributed memory settings is straightforward.

Abstract

Most numerical solvers and libraries nowadays are implemented to use mathematical models created with language-specific built-in data types (e.g. real in Fortran or double in C) and their respective elementary algebra implementations. However, the built-in elementary algebra typically has limited functionality and often restricts the flexibility of mathematical models and the analysis types that can be applied to those models. To overcome this limitation, a number of domain-specific languages such as gPROMS or Modelica with more feature-rich built-in data types have been proposed. In this paper, we argue that if numerical libraries and solvers are designed to use abstract elementary algebra rather than the language-specific built-in algebra, modern mainstream languages can be as effective as any domain-specific language. We illustrate our ideas using the example of sparse Jacobian matrix computation. We implement an automatic differentiation method that takes advantage of sparse system structures and is straightforward to parallelize in a distributed memory setting. Furthermore, we show that the computational cost scales linearly with the size of the system.