Pixel Adapter: A Graph-Based Post-Processing Approach for Scene Text Image Super-Resolution

TL;DR

This paper introduces a novel graph attention-based Pixel Adapter Module and an MLP-based residual block for scene text image super-resolution, significantly improving efficiency and recognition accuracy over existing methods.

Contribution

The paper proposes the Pixel Adapter Module with enhanced efficiency and a new residual block, along with a contour-aware loss, advancing super-resolution quality for scene text images.

Findings

Achieved 0.7% and 2.6% improvements in recognition accuracy for single-stage and multi-stage strategies.

Surpassed existing methods in recognition accuracy on TextZoom dataset.

Enhanced super-resolution quality leads to better downstream text recognition.

Abstract

Current Scene text image super-resolution approaches primarily focus on extracting robust features, acquiring text information, and complex training strategies to generate super-resolution images. However, the upsampling module, which is crucial in the process of converting low-resolution images to high-resolution ones, has received little attention in existing works. To address this issue, we propose the Pixel Adapter Module (PAM) based on graph attention to address pixel distortion caused by upsampling. The PAM effectively captures local structural information by allowing each pixel to interact with its neighbors and update features. Unlike previous graph attention mechanisms, our approach achieves 2-3 orders of magnitude improvement in efficiency and memory utilization by eliminating the dependency on sparse adjacency matrices and introducing a sliding window approach for efficient parallel computation. Additionally, we introduce the MLP-based Sequential Residual Block (MSRB) for robust feature extraction from text images, and a Local Contour Awareness loss ($\mathcal{L}_{lca}$) to enhance the model's perception of details. Comprehensive experiments on TextZoom demonstrate that our proposed method generates high-quality super-resolution images, surpassing existing methods in recognition accuracy. For single-stage and multi-stage strategies, we achieved improvements of 0.7\% and 2.6\%, respectively, increasing the performance from 52.6\% and 53.7\% to 53.3\% and 56.3\%. The code is available at https://github.com/wenyu1009/RTSRN.