Graphical Newton

TL;DR

This paper presents a method to compute the Newton step more efficiently by leveraging the known computational structure of the function, reducing the complexity from cubic in the input size to linear in the graph size.

Contribution

It introduces a novel approach to compute Newton steps in time proportional to the computational graph size and cubic in its tree-width, improving efficiency for structured functions.

Findings

Newton step computation time can be reduced to linear in graph size.

The complexity depends cubically on the tree-width of the computational graph.

The approach exploits the known structure of the function for efficiency.

Abstract

Computing the Newton step for a generic function $f: \mathbb{R}^N \rightarrow \mathbb{R}$ takes $O(N^{3})$ flops. In this paper, we explore avenues for reducing this bound, when the computational structure of $f$ is known beforehand. It is shown that the Newton step can be computed in time, linear in the size of the computational-graph, and cubic in its tree-width.