Data-Parallel Algorithms for String Diagrams

TL;DR

This paper introduces parallel algorithms for string diagram operations using a new datastructure that efficiently supports composition, tensor products, and functor applications, with applications in gradient computation and optics.

Contribution

It presents a novel datastructure enabling linear and logarithmic time parallel algorithms for string diagram manipulations, applicable to various categorical structures and gradient-based learning.

Findings

Algorithms run in linear and logarithmic time for composition and tensor operations.

The datastructure supports mapping diagrams to optics and gradient computations.

Implementation is simple, data-parallel, and adaptable to hardware and low-level languages.

Abstract

We give parallel algorithms for string diagrams represented as structured cospans of ACSets. Specifically, we give linear (sequential) and logarithmic (parallel) time algorithms for composition, tensor product, construction of diagrams from arbitrary $\Sigma$-terms, and application of functors to diagrams. Our datastructure can represent morphisms of both the free symmetric monoidal category over an arbitrary signature as well as those with a chosen Special Frobenius structure. We show how this additional (hypergraph) structure can be used to map diagrams to diagrams of optics. This leads to a case study in which we define an algorithm for efficiently computing symbolic representations of gradient-based learners based on reverse derivatives. The work we present here is intended to be useful as a general purpose datastructure. Implementation requires only integer arrays and well-known algorithms, and is data-parallel by constuction. We therefore expect it to be applicable to a wide variety of settings, including embedded and parallel hardware and low-level languages.