Equivariant Transformer Networks

TL;DR

Equivariant Transformers (ETs) are a new neural network architecture that incorporate transformation invariances directly into their structure, enhancing robustness and sample efficiency in image classification tasks.

Contribution

The paper introduces Equivariant Transformers, a novel differentiable mapping framework that embeds transformation equivariance into neural networks using canonical coordinate systems.

Findings

ETs improve model robustness to complex transformations.

ETs increase sample efficiency in limited data scenarios.

ETs achieve up to 15% error rate reduction with minimal parameter increase.

Abstract

How can prior knowledge on the transformation invariances of a domain be incorporated into the architecture of a neural network? We propose Equivariant Transformers (ETs), a family of differentiable image-to-image mappings that improve the robustness of models towards pre-defined continuous transformation groups. Through the use of specially-derived canonical coordinate systems, ETs incorporate functions that are equivariant by construction with respect to these transformations. We show empirically that ETs can be flexibly composed to improve model robustness towards more complicated transformation groups in several parameters. On a real-world image classification task, ETs improve the sample efficiency of ResNet classifiers, achieving relative improvements in error rate of up to 15% in the limited data regime while increasing model parameter count by less than 1%.