StyleTextGen: Style-Conditioned Multilingual Scene Text Generation

TL;DR

StyleTextGen is a new framework for multilingual scene text generation that improves style consistency and cross-lingual performance through a dual-branch style encoder, a style loss, and a mask-guided inference strategy.

Contribution

It introduces a novel style-conditioned text generation method with a dual-branch style encoder, style loss, and inference strategy, plus a bilingual benchmark for evaluation.

Findings

Outperforms existing methods in style consistency.

Achieves state-of-the-art in multilingual style-conditioned text generation.

Demonstrates strong cross-lingual generalization.

Abstract

Style-conditioned scene text generation faces unique challenges in extracting precise text styles from complex backgrounds and maintaining fine-grained style consistency across characters, especially for multilingual scripts. We propose StyleTextGen, a novel framework that learns to perceive and replicate visual text styles across different languages and writing systems. Our approach features three key contributions: First, we introduce a dual-branch style encoder dedicated to style modeling, yielding robust multilingual text style representations in complex real-world scenes. Second, we design a text style consistency loss that enhances style coherence and improves overall visual quality. Third, we develop a mask-guided inference strategy that ensures precise style alignment between generated and reference text. To facilitate systematic evaluation, we construct StyleText-CE, a bilingual scene text style benchmark covering both monolingual and cross-lingual settings. Extensive experiments demonstrate that StyleTextGen significantly outperforms existing methods in style consistency and cross-lingual generalization, establishing new state-of-the-art performance in multilingual style-conditioned text generation.