Convolutional layers are equivariant to discrete shifts but not continuous translations

TL;DR

This paper clarifies that convolutional layers in CNNs are shift equivariant for discrete pixel shifts but do not exhibit translation equivariance for continuous translations, emphasizing the distinction between these symmetries.

Contribution

It clarifies a common misconception by distinguishing shift equivariance from translation equivariance in convolutional layers, promoting clearer terminology.

Findings

Convolutional layers are shift equivariant to discrete pixel shifts.

They are not translation equivariant to continuous translations.

The distinction is important for understanding CNN symmetry properties.

Abstract

The purpose of this short and simple note is to clarify a common misconception about convolutional neural networks (CNNs). CNNs are made up of convolutional layers which are shift equivariant due to weight sharing. However, convolutional layers are not translation equivariant, even when boundary effects are ignored and when pooling and subsampling are absent. This is because shift equivariance is a discrete symmetry while translation equivariance is a continuous symmetry. This fact is well known among researchers in equivariant machine learning, but is usually overlooked among non-experts. To minimize confusion, we suggest using the term `shift equivariance' to refer to discrete shifts in pixels and `translation equivariance' to refer to continuous translations.