Equivariant and Steerable Neural Networks: A review with special emphasis on the symmetric group

TL;DR

This paper reviews equivariant and steerable neural networks, emphasizing the symmetric group, and details their architecture, representations, and applications in capturing symmetries beyond translation.

Contribution

It provides a comprehensive review of equivariant neural networks with a focus on the symmetric group, including new details on representations and capsules not previously documented.

Findings

Detailed architecture of equivariant layers and filter banks

Application of formalism to the symmetric group with new insights

Enhanced understanding of group representations and capsules

Abstract

Convolutional neural networks revolutionized computer vision and natrual language processing. Their efficiency, as compared to fully connected neural networks, has its origin in the architecture, where convolutions reflect the translation invariance in space and time in pattern or speech recognition tasks. Recently, Cohen and Welling have put this in the broader perspective of invariance under symmetry groups, which leads to the concept of group equivaiant neural networks and more generally steerable neural networks. In this article, we review the architecture of such networks including equivariant layers and filter banks, activation with capsules and group pooling. We apply this formalism to the symmetric group, for which we work out a number of details on representations and capsules that are not found in the literature.