Implicit Equivariance in Convolutional Networks

TL;DR

This paper introduces Implicitly Equivariant Networks (IEN), a novel approach that induces transformation equivariance in standard CNNs through a multi-objective loss, outperforming steerable CNNs in accuracy and efficiency across various tasks.

Contribution

The paper proposes IEN, a simple, flexible method to embed equivariance in CNNs without complex basis functions, enabling heterogeneous feature groups and improved performance.

Findings

IEN outperforms steerable CNNs on multiple datasets.

IEN reduces channel count by over 30% while maintaining accuracy.

IEN achieves superior results in visual object tracking.

Abstract

Convolutional Neural Networks(CNN) are inherently equivariant under translations, however, they do not have an equivalent embedded mechanism to handle other transformations such as rotations and change in scale. Several approaches exist that make CNNs equivariant under other transformation groups by design. Among these, steerable CNNs have been especially effective. However, these approaches require redesigning standard networks with filters mapped from combinations of predefined basis involving complex analytical functions. We experimentally demonstrate that these restrictions in the choice of basis can lead to model weights that are sub-optimal for the primary deep learning task (e.g. classification). Moreover, such hard-baked explicit formulations make it difficult to design composite networks comprising heterogeneous feature groups. To circumvent such issues, we propose Implicitly Equivariant Networks (IEN) which induce equivariance in the different layers of a standard CNN model by optimizing a multi-objective loss function that combines the primary loss with an equivariance loss term. Through experiments with VGG and ResNet models on Rot-MNIST , Rot-TinyImageNet, Scale-MNIST and STL-10 datasets, we show that IEN, even with its simple formulation, performs better than steerable networks. Also, IEN facilitates construction of heterogeneous filter groups allowing reduction in number of channels in CNNs by a factor of over 30% while maintaining performance on par with baselines. The efficacy of IEN is further validated on the hard problem of visual object tracking. We show that IEN outperforms the state-of-the-art rotation equivariant tracking method while providing faster inference speed.