Data Augmentation for Scene Text Recognition

TL;DR

This paper introduces STRAug, a set of 36 image augmentation functions specifically designed for scene text recognition, significantly improving model accuracy by better simulating real-world text image variations.

Contribution

The paper presents STRAug, a novel augmentation toolkit tailored for STR that enhances training data diversity and model robustness against real-world distortions.

Findings

Increases accuracy of STR models on multiple datasets

Improves robustness to noise, artifacts, and geometric distortions

Easy to implement and replicate with open-source code

Abstract

Scene text recognition (STR) is a challenging task in computer vision due to the large number of possible text appearances in natural scenes. Most STR models rely on synthetic datasets for training since there are no sufficiently big and publicly available labelled real datasets. Since STR models are evaluated using real data, the mismatch between training and testing data distributions results into poor performance of models especially on challenging text that are affected by noise, artifacts, geometry, structure, etc. In this paper, we introduce STRAug which is made of 36 image augmentation functions designed for STR. Each function mimics certain text image properties that can be found in natural scenes, caused by camera sensors, or induced by signal processing operations but poorly represented in the training dataset. When applied to strong baseline models using RandAugment, STRAug significantly increases the overall absolute accuracy of STR models across regular and irregular test datasets by as much as 2.10% on Rosetta, 1.48% on R2AM, 1.30% on CRNN, 1.35% on RARE, 1.06% on TRBA and 0.89% on GCRNN. The diversity and simplicity of API provided by STRAug functions enable easy replication and validation of existing data augmentation methods for STR. STRAug is available at https://github.com/roatienza/straug.