EAFormer: Scene Text Segmentation with Edge-Aware Transformers

TL;DR

EAFormer introduces an edge-aware transformer approach for scene text segmentation, emphasizing text edges to improve accuracy, especially at boundaries, and demonstrates superior performance on relabeled benchmarks.

Contribution

The paper proposes a novel edge-guided transformer model that explicitly incorporates text edge information for improved scene text segmentation accuracy.

Findings

Outperforms previous methods on standard benchmarks.

Achieves higher accuracy with more precise annotations.

Effectively focuses on text edges for better segmentation results.

Abstract

Scene text segmentation aims at cropping texts from scene images, which is usually used to help generative models edit or remove texts. The existing text segmentation methods tend to involve various text-related supervisions for better performance. However, most of them ignore the importance of text edges, which are significant for downstream applications. In this paper, we propose Edge-Aware Transformers, termed EAFormer, to segment texts more accurately, especially at the edge of texts. Specifically, we first design a text edge extractor to detect edges and filter out edges of non-text areas. Then, we propose an edge-guided encoder to make the model focus more on text edges. Finally, an MLP-based decoder is employed to predict text masks. We have conducted extensive experiments on commonly-used benchmarks to verify the effectiveness of EAFormer. The experimental results demonstrate that the proposed method can perform better than previous methods, especially on the segmentation of text edges. Considering that the annotations of several benchmarks (e.g., COCO_TS and MLT_S) are not accurate enough to fairly evaluate our methods, we have relabeled these datasets. Through experiments, we observe that our method can achieve a higher performance improvement when more accurate annotations are used for training.