Self-Supervised Text Erasing with Controllable Image Synthesis

TL;DR

This paper introduces an unsupervised framework for scene text erasing that synthesizes training data with style control, enabling effective text removal without costly annotations, and demonstrates superior results on a new challenging dataset.

Contribution

The paper proposes a novel self-supervised text erasing method with style-aware synthetic image generation and a triplet loss, reducing reliance on labeled data and improving erasing quality.

Findings

Achieves 5.07 FID on PosterErase, outperforming supervised methods.

Introduces a new challenging dataset with 60K high-resolution posters.

Demonstrates effectiveness of style control in synthetic data generation.

Abstract

Recent efforts on scene text erasing have shown promising results. However, existing methods require rich yet costly label annotations to obtain robust models, which limits the use for practical applications. To this end, we study an unsupervised scenario by proposing a novel Self-supervised Text Erasing (STE) framework that jointly learns to synthesize training images with erasure ground-truth and accurately erase texts in the real world. We first design a style-aware image synthesis function to generate synthetic images with diverse styled texts based on two synthetic mechanisms. To bridge the text style gap between the synthetic and real-world data, a policy network is constructed to control the synthetic mechanisms by picking style parameters with the guidance of two specifically designed rewards. The synthetic training images with erasure ground-truth are then fed to train a coarse-to-fine erasing network. To produce better erasing outputs, a triplet erasure loss is designed to enforce the refinement stage to recover background textures. Moreover, we provide a new dataset (called PosterErase), which contains 60K high-resolution posters with texts and is more challenging for the text erasing task. The proposed method has been extensively evaluated with both PosterErase and the widely-used SCUT-Enstext dataset. Notably, on PosterErase, our unsupervised method achieves 5.07 in terms of FID, with a relative performance of 20.9% over existing supervised baselines.