Scene Text Detection via Holistic, Multi-Channel Prediction

TL;DR

This paper introduces a holistic scene text detection method using a single FCN that predicts text regions, characters, and their relationships simultaneously, effectively handling various text orientations and shapes in natural images.

Contribution

The novel approach casts scene text detection as a semantic segmentation problem, leveraging multi-channel predictions in a single network for improved accuracy.

Findings

Outperforms previous state-of-the-art on ICDAR benchmarks

Handles horizontal, multi-oriented, and curved text effectively

Provides first baseline on COCO-Text dataset

Abstract

Recently, scene text detection has become an active research topic in computer vision and document analysis, because of its great importance and significant challenge. However, vast majority of the existing methods detect text within local regions, typically through extracting character, word or line level candidates followed by candidate aggregation and false positive elimination, which potentially exclude the effect of wide-scope and long-range contextual cues in the scene. To take full advantage of the rich information available in the whole natural image, we propose to localize text in a holistic manner, by casting scene text detection as a semantic segmentation problem. The proposed algorithm directly runs on full images and produces global, pixel-wise prediction maps, in which detections are subsequently formed. To better make use of the properties of text, three types of information regarding text region, individual characters and their relationship are estimated, with a single Fully Convolutional Network (FCN) model. With such predictions of text properties, the proposed algorithm can simultaneously handle horizontal, multi-oriented and curved text in real-world natural images. The experiments on standard benchmarks, including ICDAR 2013, ICDAR 2015 and MSRA-TD500, demonstrate that the proposed algorithm substantially outperforms previous state-of-the-art approaches. Moreover, we report the first baseline result on the recently-released, large-scale dataset COCO-Text.