STELLAR: Scene Text Editor for Low-Resource Languages and Real-World Data

TL;DR

STELLAR is a novel scene text editing framework that supports low-resource languages and real-world data, using a language-adaptive encoder, multi-stage training, and a new evaluation metric, TAS.

Contribution

It introduces STELLAR, a scene text editor capable of multilingual editing in low-resource settings, with a new dataset and a style preservation metric.

Findings

Outperforms state-of-the-art models in visual consistency

Achieves 2.2% higher TAS on average across languages

Demonstrates effective editing in low-resource and real-world scenarios

Abstract

Scene Text Editing (STE) is the task of modifying text content in an image while preserving its visual style, such as font, color, and background. While recent diffusion-based approaches have shown improvements in visual quality, key limitations remain: lack of support for low-resource languages, domain gap between synthetic and real data, and the absence of appropriate metrics for evaluating text style preservation. To address these challenges, we propose STELLAR (Scene Text Editor for Low-resource LAnguages and Real-world data). STELLAR enables reliable multilingual editing through a language-adaptive glyph encoder and a multi-stage training strategy that first pre-trains on synthetic data and then fine-tunes on real images. We also construct a new dataset, STIPLAR(Scene Text Image Pairs of Low-resource lAnguages and Real-world data), for training and evaluation. Furthermore, we propose Text Appearance Similarity (TAS), a novel metric that assesses style preservation by independently measuring font, color, and background similarity, enabling robust evaluation even without ground truth. Experimental results demonstrate that STELLAR outperforms state-of-the-art models in visual consistency and recognition accuracy, achieving an average TAS improvement of 2.2% across languages over the baselines.