Handheld Video Document Scanning: A Robust On-Device Model for Multi-Page Document Scanning

TL;DR

This paper introduces a robust, on-device deep learning model for handheld multi-page document scanning from video streams, enabling efficient digitization without fixed setups.

Contribution

It presents a novel on-device model, a new data collection method, and achieves state-of-the-art results in handheld multi-page document scanning.

Findings

High accuracy in multi-page document detection

Robust performance under handheld motion

State-of-the-art results on PUCIT dataset

Abstract

Document capture applications on smartphones have emerged as popular tools for digitizing documents. For many individuals, capturing documents with their smartphones is more convenient than using dedicated photocopiers or scanners, even if the quality of digitization is lower. However, using a smartphone for digitization can become excessively time-consuming and tedious when a user needs to digitize a document with multiple pages. In this work, we propose a novel approach to automatically scan multi-page documents from a video stream as the user turns through the pages of the document. Unlike previous methods that required constrained settings such as mounting the phone on a tripod, our technique is designed to allow the user to hold the phone in their hand. Our technique is trained to be robust to the motion and instability inherent in handheld scanning. Our primary contributions in this work include: (1) an efficient, on-device deep learning model that is accurate and robust for handheld scanning, (2) a novel data collection and annotation technique for video document scanning, and (3) state-of-the-art results on the PUCIT page turn dataset.