WAS: Dataset and Methods for Artistic Text Segmentation

TL;DR

This paper introduces a new dataset and methods for artistic text segmentation, addressing challenges of complex stroke shapes and global structures, and achieves state-of-the-art results through innovative model components and data synthesis strategies.

Contribution

The paper presents a novel dataset for artistic text segmentation and proposes a decoder with layer-wise momentum query, a skeleton-assisted head, and a data synthesis strategy to improve segmentation performance.

Findings

Significant performance improvement on artistic text segmentation tasks.

State-of-the-art results on multiple public datasets.

Effective data augmentation strategy enhances model generalization.

Abstract

Accurate text segmentation results are crucial for text-related generative tasks, such as text image generation, text editing, text removal, and text style transfer. Recently, some scene text segmentation methods have made significant progress in segmenting regular text. However, these methods perform poorly in scenarios containing artistic text. Therefore, this paper focuses on the more challenging task of artistic text segmentation and constructs a real artistic text segmentation dataset. One challenge of the task is that the local stroke shapes of artistic text are changeable with diversity and complexity. We propose a decoder with the layer-wise momentum query to prevent the model from ignoring stroke regions of special shapes. Another challenge is the complexity of the global topological structure. We further design a skeleton-assisted head to guide the model to focus on the global structure. Additionally, to enhance the generalization performance of the text segmentation model, we propose a strategy for training data synthesis, based on the large multi-modal model and the diffusion model. Experimental results show that our proposed method and synthetic dataset can significantly enhance the performance of artistic text segmentation and achieve state-of-the-art results on other public datasets.