Single-Character-Based Embedding Feature Aggregation Using Cross-Attention for Scene Text Super-Resolution

TL;DR

This paper introduces a new method for improving the clarity of text in images by using cross-attention to handle overlapping characters and complex backgrounds.

Contribution

The novel contribution is a single-character-based embedding feature aggregation with cross-attention for scene text super-resolution.

Findings

The proposed method improves text recognition accuracy by 0.9–1.4% over the baseline TATT on the TextZoom benchmark.

The model achieves an optimal SSIM value of 0.7951 and a PSNR of 21.84.

The approach improves accuracy by 0.2–2.2% over existing baselines on five text recognition datasets.

Abstract

In textual vision scenarios, super-resolution aims to enhance textual quality and readability to facilitate downstream tasks. However, the ambiguity of character regions in complex backgrounds remains challenging to mitigate, particularly the interference between tightly connected characters. In this paper, we propose single-character-based embedding feature aggregation using cross-attention for scene text super-resolution (SCE-STISR) to solve this problem. Firstly, a dynamic feature extraction mechanism is employed to adaptively capture shallow features by dynamically adjusting multi-scale feature weights based on spatial representations. During text–image interactions, a dual-level cross-attention mechanism is introduced to comprehensively aggregate the cropped single-character features with textual prior, also aligning semantic sequences and visual features. Finally, an adaptive normalized color correction operation is applied to mitigate color distortion caused by background interference. In TextZoom benchmarking, the text recognition accuracies of different recognizers are 53.6%, 60.9%, and 64.5%, which are improved by 0.9–1.4% over the baseline TATT, achieving an optimal SSIM value of 0.7951 and a PSNR of 21.84. Additionally, our approach improves accuracy by 0.2–2.2% over existing baselines on five text recognition datasets, validating the effectiveness of the model.

Figures