A Scene-Text Synthesis Engine Achieved Through Learning from Decomposed Real-World Data

TL;DR

This paper introduces DecompST, a real-world dataset with detailed annotations, and a learning-based engine (LBTS) that synthesizes scene text images by predicting suitable regions and adapting text appearance, improving training data quality.

Contribution

The paper presents DecompST dataset and a novel LBTS engine that learns from real-world data to generate realistic synthetic scene text images.

Findings

LBTS outperforms existing methods in generating training data.

Synthetic data from LBTS improves scene text detector performance.

DecompST provides comprehensive annotations for scene text synthesis.

Abstract

Scene-text image synthesis techniques that aim to naturally compose text instances on background scene images are very appealing for training deep neural networks due to their ability to provide accurate and comprehensive annotation information. Prior studies have explored generating synthetic text images on two-dimensional and three-dimensional surfaces using rules derived from real-world observations. Some of these studies have proposed generating scene-text images through learning; however, owing to the absence of a suitable training dataset, unsupervised frameworks have been explored to learn from existing real-world data, which might not yield reliable performance. To ease this dilemma and facilitate research on learning-based scene text synthesis, we introduce DecompST, a real-world dataset prepared from some public benchmarks, containing three types of annotations: quadrilateral-level BBoxes, stroke-level text masks, and text-erased images. Leveraging the DecompST dataset, we propose a Learning-Based Text Synthesis engine (LBTS) that includes a text location proposal network (TLPNet) and a text appearance adaptation network (TAANet). TLPNet first predicts the suitable regions for text embedding, after which TAANet adaptively adjusts the geometry and color of the text instance to match the background context. After training, those networks can be integrated and utilized to generate the synthetic dataset for scene text analysis tasks. Comprehensive experiments were conducted to validate the effectiveness of the proposed LBTS along with existing methods, and the experimental results indicate the proposed LBTS can generate better pretraining data for scene text detectors.