Learning Generative Structure Prior for Blind Text Image Super-resolution

TL;DR

This paper introduces a novel generative structure prior using StyleGAN and a codebook to improve blind text image super-resolution, especially for complex characters like Chinese, by focusing on character structures rather than recognition.

Contribution

The work proposes a structure prior based on StyleGAN and a codebook to better capture character details, enhancing super-resolution for diverse and complex text images.

Findings

Outperforms existing methods on synthetic datasets

Improves fidelity of complex character strokes

Demonstrates robustness on real-world data

Abstract

Blind text image super-resolution (SR) is challenging as one needs to cope with diverse font styles and unknown degradation. To address the problem, existing methods perform character recognition in parallel to regularize the SR task, either through a loss constraint or intermediate feature condition. Nonetheless, the high-level prior could still fail when encountering severe degradation. The problem is further compounded given characters of complex structures, e.g., Chinese characters that combine multiple pictographic or ideographic symbols into a single character. In this work, we present a novel prior that focuses more on the character structure. In particular, we learn to encapsulate rich and diverse structures in a StyleGAN and exploit such generative structure priors for restoration. To restrict the generative space of StyleGAN so that it obeys the structure of characters yet remains flexible in handling different font styles, we store the discrete features for each character in a codebook. The code subsequently drives the StyleGAN to generate high-resolution structural details to aid text SR. Compared to priors based on character recognition, the proposed structure prior exerts stronger character-specific guidance to restore faithful and precise strokes of a designated character. Extensive experiments on synthetic and real datasets demonstrate the compelling performance of the proposed generative structure prior in facilitating robust text SR.