Computational graphs for matrix functions

TL;DR

This paper introduces GraphMatFun.jl, a Julia package that models matrix functions as computational graphs, enabling optimization, automatic differentiation, and efficient code generation for more accurate and cost-effective numerical methods.

Contribution

The paper presents a novel framework for representing matrix functions as DAGs, with tools for optimization, differentiation, and code generation, improving existing numerical techniques.

Findings

Optimized polynomial-based methods can outperform Padé approximations in computational cost.

The package allows automatic differentiation of matrix functions via DAG structure.

Generated code is available for multiple programming languages, enhancing practical usability.

Abstract

Many numerical methods for evaluating matrix functions can be naturally viewed as computational graphs. Rephrasing these methods as directed acyclic graphs (DAGs) is a particularly effective approach to study existing techniques, improve them, and eventually derive new ones. The accuracy of these matrix techniques can be characterized by the accuracy of their scalar counterparts, thus designing algorithms for matrix functions can be regarded as a scalar-valued optimization problem. The derivatives needed during the optimization can be calculated automatically by exploiting the structure of the DAG, in a fashion analogous to backpropagation. This paper describes GraphMatFun.jl, a Julia package that offers the means to generate and manipulate computational graphs, optimize their coefficients, and generate Julia, MATLAB, and C code to evaluate them efficiently at a matrix argument. The software also provides tools to estimate the accuracy of a graph-based algorithm and thus obtain numerically reliable methods. For the exponential, for example, using a particular form (degree-optimal) of polynomials produces implementations that in many cases are cheaper, in terms of computational cost, than the Pad\'e-based techniques typically used in mathematical software. The optimized graphs and the corresponding generated code are available online.