TextDestroyer: A Training- and Annotation-Free Diffusion Method for Destroying Anomal Text from Images

TL;DR

TextDestroyer is a novel, training- and annotation-free diffusion-based method that effectively destroys scene text in images, enhancing privacy and content concealment without retraining or complex annotations.

Contribution

It introduces the first training- and annotation-free diffusion approach for scene text destruction, utilizing a hierarchical process and latent code manipulation for superior results.

Findings

Achieves thorough text destruction with no faint residuals

Outperforms existing methods in generalization to real-world images

Eliminates need for data annotation and retraining

Abstract

In this paper, we propose TextDestroyer, the first training- and annotation-free method for scene text destruction using a pre-trained diffusion model. Existing scene text removal models require complex annotation and retraining, and may leave faint yet recognizable text information, compromising privacy protection and content concealment. TextDestroyer addresses these issues by employing a three-stage hierarchical process to obtain accurate text masks. Our method scrambles text areas in the latent start code using a Gaussian distribution before reconstruction. During the diffusion denoising process, self-attention key and value are referenced from the original latent to restore the compromised background. Latent codes saved at each inversion step are used for replacement during reconstruction, ensuring perfect background restoration. The advantages of TextDestroyer include: (1) it eliminates labor-intensive data annotation and resource-intensive training; (2) it achieves more thorough text destruction, preventing recognizable traces; and (3) it demonstrates better generalization capabilities, performing well on both real-world scenes and generated images.