TextDiff: Mask-Guided Residual Diffusion Models for Scene Text Image Super-Resolution

TL;DR

TextDiff introduces a diffusion-based framework with a mask-guided residual module for scene text image super-resolution, significantly enhancing text clarity and recognition without extra training.

Contribution

The paper presents the first diffusion-based approach for scene text super-resolution, featuring a novel residual diffusion module that sharpens text edges effectively.

Findings

Achieves state-of-the-art performance on benchmark datasets.

Effectively sharpens text edges without additional joint training.

Improves readability and recognizability of scene text images.

Abstract

The goal of scene text image super-resolution is to reconstruct high-resolution text-line images from unrecognizable low-resolution inputs. The existing methods relying on the optimization of pixel-level loss tend to yield text edges that exhibit a notable degree of blurring, thereby exerting a substantial impact on both the readability and recognizability of the text. To address these issues, we propose TextDiff, the first diffusion-based framework tailored for scene text image super-resolution. It contains two modules: the Text Enhancement Module (TEM) and the Mask-Guided Residual Diffusion Module (MRD). The TEM generates an initial deblurred text image and a mask that encodes the spatial location of the text. The MRD is responsible for effectively sharpening the text edge by modeling the residuals between the ground-truth images and the initial deblurred images. Extensive experiments demonstrate that our TextDiff achieves state-of-the-art (SOTA) performance on public benchmark datasets and can improve the readability of scene text images. Moreover, our proposed MRD module is plug-and-play that effectively sharpens the text edges produced by SOTA methods. This enhancement not only improves the readability and recognizability of the results generated by SOTA methods but also does not require any additional joint training. Available Codes:https://github.com/Lenubolim/TextDiff.