Invariant Integration in Deep Convolutional Feature Space

TL;DR

This paper introduces a novel invariant integration layer for deep neural networks that enforces feature space invariances, improving classification performance especially with limited data, demonstrated on rotated image datasets.

Contribution

We propose a new layer based on invariant integration to embed invariances into deep neural networks, enabling explicit handling of transformation groups.

Findings

Achieved state-of-the-art results on Rotated-MNIST

Demonstrated improved performance with limited training data

Effectively enforced rotation invariance in deep features

Abstract

In this contribution, we show how to incorporate prior knowledge to a deep neural network architecture in a principled manner. We enforce feature space invariances using a novel layer based on invariant integration. This allows us to construct a complete feature space invariant to finite transformation groups. We apply our proposed layer to explicitly insert invariance properties for vision-related classification tasks, demonstrate our approach for the case of rotation invariance and report state-of-the-art performance on the Rotated-MNIST dataset. Our method is especially beneficial when training with limited data.