Words as Geometric Features: Estimating Homography using Optical Character Recognition as Compressed Image Representation

TL;DR

This paper presents a novel OCR-based method for estimating homographies in document alignment, enabling effective registration using only textual features and spatial positions, especially useful when image data is unavailable.

Contribution

The paper introduces a new approach that uses OCR outputs as geometric features for homography estimation, bypassing the need for raw image data.

Findings

OCR-based method outperforms traditional image-based approaches in accuracy.

The approach is robust to OCR noise and outliers.

It offers a scalable solution for document registration when images are inaccessible.

Abstract

Document alignment and registration play a crucial role in numerous real-world applications, such as automated form processing, anomaly detection, and workflow automation. Traditional methods for document alignment rely on image-based features like keypoints, edges, and textures to estimate geometric transformations, such as homographies. However, these approaches often require access to the original document images, which may not always be available due to privacy, storage, or transmission constraints. This paper introduces a novel approach that leverages Optical Character Recognition (OCR) outputs as features for homography estimation. By utilizing the spatial positions and textual content of OCR-detected words, our method enables document alignment without relying on pixel-level image data. This technique is particularly valuable in scenarios where only OCR outputs are accessible. Furthermore, the method is robust to OCR noise, incorporating RANSAC to handle outliers and inaccuracies in the OCR data. On a set of test documents, we demonstrate that our OCR-based approach even performs more accurately than traditional image-based methods, offering a more efficient and scalable solution for document registration tasks. The proposed method facilitates applications in document processing, all while reducing reliance on high-dimensional image data.