Transform-Invariant Convolutional Neural Networks for Image Classification and Search

TL;DR

This paper introduces a method where CNN feature maps are randomly transformed during training to enhance invariance to spatial transformations like rotation, scale, and translation, improving performance on various image recognition tasks.

Contribution

The paper proposes a simple, supervision-free technique of random feature map transformations during training to achieve transform invariance in CNNs.

Findings

Significant improvements on benchmark image recognition tasks.

Enhanced robustness to spatial transformations without extra supervision.

Applicable to small-scale, large-scale, and retrieval tasks.

Abstract

Convolutional neural networks (CNNs) have achieved state-of-the-art results on many visual recognition tasks. However, current CNN models still exhibit a poor ability to be invariant to spatial transformations of images. Intuitively, with sufficient layers and parameters, hierarchical combinations of convolution (matrix multiplication and non-linear activation) and pooling operations should be able to learn a robust mapping from transformed input images to transform-invariant representations. In this paper, we propose randomly transforming (rotation, scale, and translation) feature maps of CNNs during the training stage. This prevents complex dependencies of specific rotation, scale, and translation levels of training images in CNN models. Rather, each convolutional kernel learns to detect a feature that is generally helpful for producing the transform-invariant answer given the combinatorially large variety of transform levels of its input feature maps. In this way, we do not require any extra training supervision or modification to the optimization process and training images. We show that random transformation provides significant improvements of CNNs on many benchmark tasks, including small-scale image recognition, large-scale image recognition, and image retrieval. The code is available at https://github.com/jasonustc/caffe-multigpu/tree/TICNN.