Compensating for Large In-Plane Rotations in Natural Images

TL;DR

This paper introduces a novel approach to achieve robustness against large in-plane rotations in images by training a CNN to detect rotations, using Bayesian Optimization to find unrotated images, and applying this as a pre-processing step for improved image retrieval.

Contribution

The paper presents a new method combining a specialized CNN, Bayesian Optimization, and a pre-processing pipeline to handle large in-plane rotations, surpassing traditional invariant feature approaches.

Findings

CNN with convolutional template layer effectively detects rotations

Bayesian Optimization efficiently finds unrotated images

Method improves robustness in image retrieval tasks

Abstract

Rotation invariance has been studied in the computer vision community primarily in the context of small in-plane rotations. This is usually achieved by building invariant image features. However, the problem of achieving invariance for large rotation angles remains largely unexplored. In this work, we tackle this problem by directly compensating for large rotations, as opposed to building invariant features. This is inspired by the neuro-scientific concept of mental rotation, which humans use to compare pairs of rotated objects. Our contributions here are three-fold. First, we train a Convolutional Neural Network (CNN) to detect image rotations. We find that generic CNN architectures are not suitable for this purpose. To this end, we introduce a convolutional template layer, which learns representations for canonical 'unrotated' images. Second, we use Bayesian Optimization to quickly sift through a large number of candidate images to find the canonical 'unrotated' image. Third, we use this method to achieve robustness to large angles in an image retrieval scenario. Our method is task-agnostic, and can be used as a pre-processing step in any computer vision system.