SEE: Towards Semi-Supervised End-to-End Scene Text Recognition

TL;DR

This paper introduces SEE, a semi-supervised end-to-end neural network that jointly detects and recognizes scene text in images, simplifying the process with a single model and achieving competitive results.

Contribution

The paper presents a novel semi-supervised, end-to-end neural network that combines text detection and recognition in a single architecture, reducing complexity compared to prior multi-network systems.

Findings

Achieves competitive results on standard benchmarks.

Demonstrates feasibility of semi-supervised end-to-end training.

Integrates spatial transformer and recognition networks effectively.

Abstract

Detecting and recognizing text in natural scene images is a challenging, yet not completely solved task. In recent years several new systems that try to solve at least one of the two sub-tasks (text detection and text recognition) have been proposed. In this paper we present SEE, a step towards semi-supervised neural networks for scene text detection and recognition, that can be optimized end-to-end. Most existing works consist of multiple deep neural networks and several pre-processing steps. In contrast to this, we propose to use a single deep neural network, that learns to detect and recognize text from natural images, in a semi-supervised way. SEE is a network that integrates and jointly learns a spatial transformer network, which can learn to detect text regions in an image, and a text recognition network that takes the identified text regions and recognizes their textual content. We introduce the idea behind our novel approach and show its feasibility, by performing a range of experiments on standard benchmark datasets, where we achieve competitive results.