Why my photos look sideways or upside down? Detecting Canonical Orientation of Images using Convolutional Neural Networks

TL;DR

This paper introduces a deep learning approach using pre-trained CNNs to accurately detect the canonical orientation of images, outperforming previous methods and approaching human-level accuracy.

Contribution

It is the first to leverage large-scale pre-trained CNNs and extensive datasets for image orientation detection, significantly improving accuracy over existing low-level feature methods.

Findings

Outperforms state-of-the-art methods in orientation detection accuracy.

Generalizes well across various public datasets.

Achieves accuracy close to human performance.

Abstract

Image orientation detection requires high-level scene understanding. Humans use object recognition and contextual scene information to correctly orient images. In literature, the problem of image orientation detection is mostly confronted by using low-level vision features, while some approaches incorporate few easily detectable semantic cues to gain minor improvements. The vast amount of semantic content in images makes orientation detection challenging, and therefore there is a large semantic gap between existing methods and human behavior. Also, existing methods in literature report highly discrepant detection rates, which is mainly due to large differences in datasets and limited variety of test images used for evaluation. In this work, for the first time, we leverage the power of deep learning and adapt pre-trained convolutional neural networks using largest training dataset to-date for the image orientation detection task. An extensive evaluation of our model on different public datasets shows that it remarkably generalizes to correctly orient a large set of unconstrained images; it also significantly outperforms the state-of-the-art and achieves accuracy very close to that of humans.