PEAN: A Diffusion-Based Prior-Enhanced Attention Network for Scene Text Image Super-Resolution

TL;DR

PEAN introduces a diffusion-based prior-enhanced attention network that significantly improves scene text image super-resolution by integrating local-global dependence understanding and semantic text priors, achieving state-of-the-art results.

Contribution

The paper proposes a novel PEAN model combining attention modulation and diffusion-based text priors for enhanced scene text image super-resolution.

Findings

PEAN achieves new SOTA on TextZoom benchmark.

The diffusion-based text prior significantly improves SR quality.

Multi-task learning enhances model performance.

Abstract

Scene text image super-resolution (STISR) aims at simultaneously increasing the resolution and readability of low-resolution scene text images, thus boosting the performance of the downstream recognition task. Two factors in scene text images, visual structure and semantic information, affect the recognition performance significantly. To mitigate the effects from these factors, this paper proposes a Prior-Enhanced Attention Network (PEAN). Specifically, an attention-based modulation module is leveraged to understand scene text images by neatly perceiving the local and global dependence of images, despite the shape of the text. Meanwhile, a diffusion-based module is developed to enhance the text prior, hence offering better guidance for the SR network to generate SR images with higher semantic accuracy. Additionally, a multi-task learning paradigm is employed to optimize the network, enabling the model to generate legible SR images. As a result, PEAN establishes new SOTA results on the TextZoom benchmark. Experiments are also conducted to analyze the importance of the enhanced text prior as a means of improving the performance of the SR network. Code is available at https://github.com/jdfxzzy/PEAN.