Let's Enhance: A Deep Learning Approach to Extreme Deblurring of Text Images

TL;DR

This paper introduces a deep learning pipeline for extreme text image deblurring, utilizing synthetic data augmentation and physical model estimation, achieving state-of-the-art OCR accuracy in a challenging real-world dataset.

Contribution

The authors develop a novel deep learning approach combining physical model estimation, synthetic data augmentation, and end-to-end training for text image deblurring, winning the Helsinki Deblur Challenge 2021.

Findings

Achieved over 70% OCR accuracy on the hardest challenge level.

Demonstrated the effectiveness of data-centric machine learning for inverse problems.

Validated the importance of physical forward model estimation and synthetic data augmentation.

Abstract

This work presents a novel deep-learning-based pipeline for the inverse problem of image deblurring, leveraging augmentation and pre-training with synthetic data. Our results build on our winning submission to the recent Helsinki Deblur Challenge 2021, whose goal was to explore the limits of state-of-the-art deblurring algorithms in a real-world data setting. The task of the challenge was to deblur out-of-focus images of random text, thereby in a downstream task, maximizing an optical-character-recognition-based score function. A key step of our solution is the data-driven estimation of the physical forward model describing the blur process. This enables a stream of synthetic data, generating pairs of ground-truth and blurry images on-the-fly, which is used for an extensive augmentation of the small amount of challenge data provided. The actual deblurring pipeline consists of an approximate inversion of the radial lens distortion (determined by the estimated forward model) and a U-Net architecture, which is trained end-to-end. Our algorithm was the only one passing the hardest challenge level, achieving over $70\%$ character recognition accuracy. Our findings are well in line with the paradigm of data-centric machine learning, and we demonstrate its effectiveness in the context of inverse problems. Apart from a detailed presentation of our methodology, we also analyze the importance of several design choices in a series of ablation studies. The code of our challenge submission is available under https://github.com/theophil-trippe/HDC_TUBerlin_version_1.