What is an equivariant neural network?

TL;DR

This paper clarifies the mathematical foundations of equivariant neural networks, highlighting their importance in breakthroughs like CNNs and AlphaFold 2, while briefly addressing engineering challenges.

Contribution

It provides a clear mathematical explanation of equivariant neural networks, emphasizing their fundamental role in recent machine learning advances.

Findings

Equivariant neural networks are key to recent breakthroughs in ML.

Mathematical ideas behind equivariance are simple but often obscured.

Engineering challenges are secondary to the core mathematical concepts.

Abstract

We explain equivariant neural networks, a notion underlying breakthroughs in machine learning from deep convolutional neural networks for computer vision to AlphaFold 2 for protein structure prediction, without assuming knowledge of equivariance or neural networks. The basic mathematical ideas are simple but are often obscured by engineering complications that come with practical realizations. We extract and focus on the mathematical aspects, and limit ourselves to a cursory treatment of the engineering issues at the end.