Stochastic Neural Network Symmetrisation in Markov Categories

TL;DR

This paper introduces a flexible framework using Markov categories to symmetrise neural networks along group homomorphisms, accommodating stochastic outputs and extending existing deterministic methods.

Contribution

It develops a novel, compositional approach for symmetrising neural networks, including stochastic models, based on minimal assumptions about group structure and architecture.

Findings

Unified framework for deterministic and stochastic symmetrisation

Extension of canonicalisation techniques to stochastic models

Demonstrates utility of Markov categories in machine learning

Abstract

We consider the problem of symmetrising a neural network along a group homomorphism: given a homomorphism $\varphi : H \to G$, we would like a procedure that converts $H$-equivariant neural networks to $G$-equivariant ones. We formulate this in terms of Markov categories, which allows us to consider neural networks whose outputs may be stochastic, but with measure-theoretic details abstracted away. We obtain a flexible and compositional framework for symmetrisation that relies on minimal assumptions about the structure of the group and the underlying neural network architecture. Our approach recovers existing canonicalisation and averaging techniques for symmetrising deterministic models, and extends to provide a novel methodology for symmetrising stochastic models also. Beyond this, our findings also demonstrate the utility of Markov categories for addressing complex problems in machine learning in a conceptually clear yet mathematically precise way.