TPSNet: Reverse Thinking of Thin Plate Splines for Arbitrary Shape Scene Text Representation

TL;DR

TPSNet introduces a novel scene text detection method using Thin-Plate-Spline (TPS) parameters as a compact, complete, and efficient shape representation, improving arbitrary shape text detection and recognition accuracy.

Contribution

The paper proposes using TPS as a new shape representation for scene text, along with Border Alignment Loss, enabling direct rectification and improved detection and spotting performance.

Findings

Achieves 4.4% higher F-Measure on Art dataset

Achieves 5.0% higher end-to-end spotting F-Measure on Total-Text

Demonstrates effectiveness and superiority over existing methods

Abstract

The research focus of scene text detection and recognition has shifted to arbitrary shape text in recent years, where the text shape representation is a fundamental problem. An ideal representation should be compact, complete, efficient, and reusable for subsequent recognition in our opinion. However, previous representations have flaws in one or more aspects. Thin-Plate-Spline (TPS) transformation has achieved great success in scene text recognition. Inspired by this, we reversely think of its usage and sophisticatedly take TPS as an exquisite representation for arbitrary shape text representation. The TPS representation is compact, complete, and efficient. With the predicted TPS parameters, the detected text region can be directly rectified to a near-horizontal one to assist the subsequent recognition. To further exploit the potential of the TPS representation, the Border Alignment Loss is proposed. Based on these designs, we implement the text detector TPSNet, which can be extended to a text spotter conveniently. Extensive evaluation and ablation of several public benchmarks demonstrate the effectiveness and superiority of the proposed method for text representation and spotting. Particularly, TPSNet achieves the detection F-Measure improvement of 4.4\% (78.4\% vs. 74.0\%) on Art dataset and the end-to-end spotting F-Measure improvement of 5.0\% (78.5\% vs. 73.5\%) on Total-Text, which are large margins with no bells and whistles.