Deep Image Homography Estimation

TL;DR

This paper introduces a deep learning approach for estimating image homographies directly from image pairs, eliminating the need for traditional feature detection and matching, and demonstrating superior performance in certain scenarios.

Contribution

The paper proposes a novel deep convolutional neural network architecture for homography estimation, including both regression and classification models, trained end-to-end on warped MS-COCO images.

Findings

Deep models outperform traditional ORB-based methods in specific scenarios.

The approach simplifies homography estimation by removing feature detection steps.

Applications demonstrate the versatility of deep homography estimation.

Abstract

We present a deep convolutional neural network for estimating the relative homography between a pair of images. Our feed-forward network has 10 layers, takes two stacked grayscale images as input, and produces an 8 degree of freedom homography which can be used to map the pixels from the first image to the second. We present two convolutional neural network architectures for HomographyNet: a regression network which directly estimates the real-valued homography parameters, and a classification network which produces a distribution over quantized homographies. We use a 4-point homography parameterization which maps the four corners from one image into the second image. Our networks are trained in an end-to-end fashion using warped MS-COCO images. Our approach works without the need for separate local feature detection and transformation estimation stages. Our deep models are compared to a traditional homography estimator based on ORB features and we highlight the scenarios where HomographyNet outperforms the traditional technique. We also describe a variety of applications powered by deep homography estimation, thus showcasing the flexibility of a deep learning approach.