A Functorial Formulation of Neighborhood Aggregating Deep Learning

TL;DR

This paper offers a mathematical framework for understanding convolutional neural networks using topology and category theory, revealing limitations through sheaf-theoretic obstructions.

Contribution

It introduces a functorial, topological perspective on neural networks, connecting their structure to sheaf theory and identifying theoretical limitations.

Findings

Provides a presheaf and copresheaf interpretation of neural networks

Formulates obstructions as limitations of neural network expressiveness

Offers a new mathematical heuristic for analyzing neural network capabilities

Abstract

We provide a mathematical interpretation of convolutional (or message passing) neural networks by using presheaves and copresheaves of the set of continuous functions over a topological space. Based on this interpretation, we formulate a theoretical heuristic which elaborates a number of empirical limitations of these neural networks by using obstructions on such sets of continuous functions over a topological space to be sheaves or copresheaves.