Using and Abusing Equivariance

TL;DR

This paper examines how subsampling in Group Equivariant CNNs can break their symmetry properties, affecting generalization, and shows that approximate equivariance can sometimes outperform exact equivariance when training data symmetries differ.

Contribution

It reveals how small input changes can cause networks to become approximately equivariant and demonstrates the conditions under which approximate equivariance can be beneficial.

Findings

Approximately equivariant networks generalize worse to unseen symmetries.

Exact equivariance leads to better symmetry generalization.

Approximate equivariance can outperform exact when training symmetries differ.

Abstract

In this paper we show how Group Equivariant Convolutional Neural Networks use subsampling to learn to break equivariance to their symmetries. We focus on 2D rotations and reflections and investigate the impact of broken equivariance on network performance. We show that a change in the input dimension of a network as small as a single pixel can be enough for commonly used architectures to become approximately equivariant, rather than exactly. We investigate the impact of networks not being exactly equivariant and find that approximately equivariant networks generalise significantly worse to unseen symmetries compared to their exactly equivariant counterparts. However, when the symmetries in the training data are not identical to the symmetries of the network, we find that approximately equivariant networks are able to relax their own equivariant constraints, causing them to match or outperform exactly equivariant networks on common benchmark datasets.