On the Computational Complexity of the Chain Rule of Differential Calculus

TL;DR

This paper investigates the computational complexity of applying the chain rule in differentiation, proving that the problem of optimizing derivative computations in composite functions is NP-complete, highlighting fundamental limitations in computational efficiency.

Contribution

The paper formalizes the Chain Rule Differentiation problem and proves its NP-completeness, providing a theoretical foundation for understanding its computational difficulty.

Findings

Proves Chain Rule Differentiation is NP-complete

Highlights the combinatorial complexity in derivative computations

Discusses implications for numerical methods in science and engineering

Abstract

Many modern numerical methods in computational science and engineering rely on derivatives of mathematical models for the phenomena under investigation. The computation of these derivatives often represents the bottleneck in terms of overall runtime performance. First and higher derivative tensors need to be evaluated efficiently. The chain rule of differentiation is the fundamental prerequisite for computing accurate derivatives of composite functions which perform a potentially very large number of elemental function evaluations. Data flow dependences amongst the elemental functions give rise to a combinatorial optimization problem. We formulate {\sc Chain Rule Differentiation} and we prove it to be NP-complete. Pointers to research on its approximate solution are given.