Fast Projective Image Rectification for Planar Objects with Manhattan Structure

TL;DR

This paper introduces a fast, accurate method for rectifying images of planar objects with Manhattan structure by estimating vanishing points and camera rotation, applicable to various objects including documents and facades.

Contribution

The paper proposes a novel optimization-based approach for vanishing point estimation and a new rectification method based on camera rotation estimation, achieving real-time performance.

Findings

Accuracy comparable or superior to state-of-the-art methods for images with limited background complexity.

Runtime of approximately 3 milliseconds on a standard CPU.

Effective for rectifying diverse planar objects like documents and building facades.

Abstract

This paper presents a method for metric rectification of planar objects that preserves angles and length ratios. An inner structure of an object is assumed to follow the laws of Manhattan World i.e. the majority of line segments are aligned with two orthogonal directions of the object. For that purpose we introduce the method that estimates the position of two vanishing points corresponding to the main object directions. It is based on an original optimization function of segments that estimates a vanishing point position. For calculation of the rectification homography with two vanishing points we propose a new method based on estimation of the camera rotation so that the camera axis is perpendicular to the object plane. The proposed method can be applied for rectification of various objects such as documents or building facades. Also since the camera rotation is estimated the method can be employed for estimation of object orientation (for example, during a surgery with radiograph of osteosynthesis implants). The method was evaluated on the MIDV-500 dataset containing projectively distorted images of documents with complex background. According to the experimental results an accuracy of the proposed method is better or equal to the-state-of-the-art if the background occupies no more than half of the image. Runtime of the method is around 3ms on core i7 3610qm CPU.