SELECT: Detecting Label Errors in Real-world Scene Text Data

TL;DR

SELECT is a novel multi-modal approach that effectively detects label errors in real-world scene text datasets, addressing challenges like variable-length labels and character errors, and improving scene text recognition accuracy.

Contribution

Introduces SELECT, a multi-modal method with SSLC for realistic label error detection in scene text data, handling variable-length labels and character similarities.

Findings

SELECT outperforms existing methods in accuracy.

Detects label errors in real-world datasets effectively.

Improves scene text recognition performance.

Abstract

We introduce SELECT (Scene tExt Label Errors deteCTion), a novel approach that leverages multi-modal training to detect label errors in real-world scene text datasets. Utilizing an image-text encoder and a character-level tokenizer, SELECT addresses the issues of variable-length sequence labels, label sequence misalignment, and character-level errors, outperforming existing methods in accuracy and practical utility. In addition, we introduce Similarity-based Sequence Label Corruption (SSLC), a process that intentionally introduces errors into the training labels to mimic real-world error scenarios during training. SSLC not only can cause a change in the sequence length but also takes into account the visual similarity between characters during corruption. Our method is the first to detect label errors in real-world scene text datasets successfully accounting for variable-length labels. Experimental results demonstrate the effectiveness of SELECT in detecting label errors and improving STR accuracy on real-world text datasets, showcasing its practical utility.