Numerical Stability of Tangents and Adjoints of Implicit Functions

TL;DR

This paper analyzes the numerical stability of tangents and adjoints of implicit functions, revealing stability conditions and highlighting differences between linear and nonlinear systems.

Contribution

It provides a comprehensive analysis of errors in tangents and adjoints, establishing stability conditions for various types of systems and extending previous understanding.

Findings

Adjoints of linear systems are unconditionally stable.

Tangents of linear systems can become unstable.

Stability conditions for nonlinear systems are derived.

Abstract

We investigate errors in tangents and adjoints of implicit functions resulting from errors in the primal solution due to approximations computed by a numerical solver. Adjoints of systems of linear equations turn out to be unconditionally numerically stable. Tangents of systems of linear equations can become instable as well as both tangents and adjoints of systems of nonlinear equations, which extends to optima of convex unconstrained objectives. Sufficient conditions for numerical stability are derived.