Graph Convolutional Neural Networks as Parametric CoKleisli morphisms

TL;DR

This paper develops a categorical framework for Graph Convolutional Neural Networks (GCNNs), representing their structure as parametric CoKleisli morphisms, which offers a high-level understanding of their inductive biases and potential generalizations.

Contribution

It introduces a bicategory of GCNNs, factors it through existing categorical constructions, and characterizes adjacency matrices as global parameters, providing new theoretical insights.

Findings

Categorical characterization of GCNNs as parametric CoKleisli morphisms

Adjacency matrices treated as global parameters rather than local layers

High-level categorical understanding of inductive biases in GCNNs

Abstract

We define the bicategory of Graph Convolutional Neural Networks $\mathbf{GCNN}_n$ for an arbitrary graph with $n$ nodes. We show it can be factored through the already existing categorical constructions for deep learning called $\mathbf{Para}$ and $\mathbf{Lens}$ with the base category set to the CoKleisli category of the product comonad. We prove that there exists an injective-on-objects, faithful 2-functor $\mathbf{GCNN}_n \to \mathbf{Para}(\mathsf{CoKl}(\mathbb{R}^{n \times n} \times -))$. We show that this construction allows us to treat the adjacency matrix of a GCNN as a global parameter instead of a a local, layer-wise one. This gives us a high-level categorical characterisation of a particular kind of inductive bias GCNNs possess. Lastly, we hypothesize about possible generalisations of GCNNs to general message-passing graph neural networks, connections to equivariant learning, and the (lack of) functoriality of activation functions.