Notes on Computational Graph and Jacobian Accumulation

TL;DR

This paper explores the relationship between computational graphs and algebraic expressions, revealing their interconvertibility and establishing theoretical limits on calculation efficiency through analysis of multiplication relations and elimination dependencies.

Contribution

It provides a theoretical framework linking computational graphs and algebraic expressions, clarifying their relations and limitations in optimizing calculation order.

Findings

Identifies the close relation between computational graphs and algebraic expressions.

Reveals different multiplication relations and their impact on elimination dependencies.

Establishes a theoretical limit on face elimination efficiency.

Abstract

The optimal calculation order of a computational graph can be represented by a set of algebraic expressions. Computational graph and algebraic expression both have close relations and significant differences, this paper looks into these relations and differences, making plain their interconvertibility. By revealing different types of multiplication relations in algebraic expressions and their elimination dependencies in line-graph, we establish a theoretical limit on the efficiency of face elimination.