Revisiting Tampered Scene Text Detection in the Era of Generative AI

TL;DR

This paper introduces a new open-set tampered scene text detection task, a comprehensive dataset, and a novel training framework that significantly improves the detection of both seen and unseen text forgeries, addressing current generalization limitations.

Contribution

The paper proposes a new open-set detection task, curates a high-quality dataset with multiple forgery types, and introduces DAF, a framework that enhances generalization by focusing on authentic versus tampered features.

Findings

Zero-shot detection outperforms previous full-shot models.

Texture alteration training improves fine-grained perception.

The dataset enables robust evaluation of unseen forgery types.

Abstract

The rapid advancements of generative AI have fueled the potential of generative text image editing, meanwhile escalating the threat of misinformation spreading. However, existing forensics methods struggle to detect unseen forgery types that they have not been trained on, underscoring the need for a model capable of generalized detection of tampered scene text. To tackle this, we propose a novel task: open-set tampered scene text detection, which evaluates forensics models on their ability to identify both seen and previously unseen forgery types. We have curated a comprehensive, high-quality dataset, featuring the texts tampered by eight text editing models, to thoroughly assess the open-set generalization capabilities. Further, we introduce a novel and effective training paradigm that subtly alters the texture of selected texts within an image and trains the model to identify these regions. This approach not only mitigates the scarcity of high-quality training data but also enhances models' fine-grained perception and open-set generalization abilities. Additionally, we present DAF, a novel framework that improves open-set generalization by distinguishing between the features of authentic and tampered text, rather than focusing solely on the tampered text's features. Our extensive experiments validate the remarkable efficacy of our methods. For example, our zero-shot performance can even beat the previous state-of-the-art full-shot model by a large margin. Our dataset and code are available at https://github.com/qcf-568/OSTF.