TI-POOLING: transformation-invariant pooling for feature learning in Convolutional Neural Networks

TL;DR

This paper introduces TI-POOLING, a transformation-invariant pooling operator for CNNs, enabling the network to handle nuisance variations like rotation and scale more efficiently than traditional data augmentation methods.

Contribution

The paper proposes a novel transformation-invariant pooling operator integrated into CNNs, reducing the need for extensive data augmentation and improving performance with fewer parameters.

Findings

Improved accuracy on benchmark datasets

Fewer parameters needed compared to augmented models

Efficient handling of transformation variations

Abstract

In this paper we present a deep neural network topology that incorporates a simple to implement transformation invariant pooling operator (TI-POOLING). This operator is able to efficiently handle prior knowledge on nuisance variations in the data, such as rotation or scale changes. Most current methods usually make use of dataset augmentation to address this issue, but this requires larger number of model parameters and more training data, and results in significantly increased training time and larger chance of under- or overfitting. The main reason for these drawbacks is that the learned model needs to capture adequate features for all the possible transformations of the input. On the other hand, we formulate features in convolutional neural networks to be transformation-invariant. We achieve that using parallel siamese architectures for the considered transformation set and applying the TI-POOLING operator on their outputs before the fully-connected layers. We show that this topology internally finds the most optimal "canonical" instance of the input image for training and therefore limits the redundancy in learned features. This more efficient use of training data results in better performance on popular benchmark datasets with smaller number of parameters when comparing to standard convolutional neural networks with dataset augmentation and to other baselines.