SPTS: Single-Point Text Spotting

TL;DR

This paper introduces a novel scene text spotting method that requires only a single point annotation per text instance, significantly reducing annotation costs while achieving state-of-the-art results using an auto-regressive Transformer model.

Contribution

It presents the first approach to train scene text spotting models with only single-point annotations, simplifying data labeling and maintaining high performance.

Findings

Achieves state-of-the-art results on benchmark datasets.

Performance is robust to the position of point annotations.

Reduces annotation cost significantly compared to bounding box methods.

Abstract

Existing scene text spotting (i.e., end-to-end text detection and recognition) methods rely on costly bounding box annotations (e.g., text-line, word-level, or character-level bounding boxes). For the first time, we demonstrate that training scene text spotting models can be achieved with an extremely low-cost annotation of a single-point for each instance. We propose an end-to-end scene text spotting method that tackles scene text spotting as a sequence prediction task. Given an image as input, we formulate the desired detection and recognition results as a sequence of discrete tokens and use an auto-regressive Transformer to predict the sequence. The proposed method is simple yet effective, which can achieve state-of-the-art results on widely used benchmarks. Most significantly, we show that the performance is not very sensitive to the positions of the point annotation, meaning that it can be much easier to be annotated or even be automatically generated than the bounding box that requires precise positions. We believe that such a pioneer attempt indicates a significant opportunity for scene text spotting applications of a much larger scale than previously possible. The code is available at https://github.com/shannanyinxiang/SPTS.