Diffusion-based Blind Text Image Super-Resolution

TL;DR

This paper introduces a diffusion-based approach for blind text image super-resolution, effectively restoring degraded low-resolution text images with accurate structure and realistic appearance, especially for complex Chinese characters.

Contribution

It proposes a novel combination of text and image diffusion models with a multi-modality module to enhance text image super-resolution quality.

Findings

Improved text structure accuracy in super-resolved images

Enhanced realism in restored text images

Effective handling of complex Chinese characters

Abstract

Recovering degraded low-resolution text images is challenging, especially for Chinese text images with complex strokes and severe degradation in real-world scenarios. Ensuring both text fidelity and style realness is crucial for high-quality text image super-resolution. Recently, diffusion models have achieved great success in natural image synthesis and restoration due to their powerful data distribution modeling abilities and data generation capabilities. In this work, we propose an Image Diffusion Model (IDM) to restore text images with realistic styles. For diffusion models, they are not only suitable for modeling realistic image distribution but also appropriate for learning text distribution. Since text prior is important to guarantee the correctness of the restored text structure according to existing arts, we also propose a Text Diffusion Model (TDM) for text recognition which can guide IDM to generate text images with correct structures. We further propose a Mixture of Multi-modality module (MoM) to make these two diffusion models cooperate with each other in all the diffusion steps. Extensive experiments on synthetic and real-world datasets demonstrate that our Diffusion-based Blind Text Image Super-Resolution (DiffTSR) can restore text images with more accurate text structures as well as more realistic appearances simultaneously.