Categorical Equivariant Deep Learning: Category-Equivariant Neural Networks and Universal Approximation Theorems

TL;DR

This paper introduces a comprehensive categorical framework for equivariant neural networks, unifying various symmetry types and proving their universal approximation capabilities across multiple structures.

Contribution

It develops a general theory of category-equivariant neural networks, extending equivariance beyond groups to posets, graphs, and sheaves, with universal approximation theorems.

Findings

Unified categorical framework for equivariant neural networks

Universal approximation theorems for various structures

Expansion of equivariance to contextual and compositional symmetries

Abstract

We develop a theory of category-equivariant neural networks (CENNs) that unifies group/groupoid-equivariant networks, poset/lattice-equivariant networks, graph and sheaf neural networks. Equivariance is formulated as naturality in a topological category with Radon measures. Formulating linear and nonlinear layers in the categorical setup, we prove the equivariant universal approximation theorem in the general setting: the class of finite-depth CENNs is dense in the space of continuous equivariant transformations. We instantiate the framework for groups/groupoids, posets/lattices, graphs and cellular sheaves, deriving universal approximation theorems for them in a systematic manner. Categorical equivariant deep learning thus allows us to expand the horizons of equivariant deep learning beyond group actions, encompassing not only geometric symmetries but also contextual and compositional symmetries.