Interrelation of equivariant Gaussian processes and convolutional neural networks

TL;DR

This paper explores the connection between equivariant Gaussian processes and convolutional neural networks, particularly focusing on their symmetry properties and the relationship in the many-channel limit.

Contribution

It establishes a theoretical link between equivariant CNNs and equivariant Gaussian processes, advancing understanding of their relationship in the context of symmetry and neural activations.

Findings

Derived the relationship between equivariant CNNs and Gaussian processes.

Analyzed the many-channel limit for equivariant CNNs.

Connected neural network equivariance with Gaussian process models.

Abstract

Currently there exists rather promising new trend in machine leaning (ML) based on the relationship between neural networks (NN) and Gaussian processes (GP), including many related subtopics, e.g., signal propagation in NNs, theoretical derivation of learning curve for NNs, QFT methods in ML, etc. An important feature of convolutional neural networks (CNN) is their equivariance (consistency) with respect to the symmetry transformations of the input data. In this work we establish a relationship between the many-channel limit for CNNs equivariant with respect to two-dimensional Euclidean group with vector-valued neuron activations and the corresponding independently introduced equivariant Gaussian processes (GP).