Improving Scene Text Recognition for Character-Level Long-Tailed Distribution

TL;DR

This paper introduces CAFE-Net, a novel approach for scene text recognition that employs two specialized experts to handle long-tailed character distributions, improving recognition accuracy for languages with many characters.

Contribution

The paper proposes a dual-expert framework with a confidence ensemble method to enhance scene text recognition in long-tailed character distributions, addressing limitations of existing models.

Findings

CAFE-Net improves recognition accuracy on long-tailed character datasets.

The dual-expert approach effectively balances contextual and visual information.

The method is adaptable to various scene text recognition models.

Abstract

Despite the recent remarkable improvements in scene text recognition (STR), the majority of the studies focused mainly on the English language, which only includes few number of characters. However, STR models show a large performance degradation on languages with a numerous number of characters (e.g., Chinese and Korean), especially on characters that rarely appear due to the long-tailed distribution of characters in such languages. To address such an issue, we conducted an empirical analysis using synthetic datasets with different character-level distributions (e.g., balanced and long-tailed distributions). While increasing a substantial number of tail classes without considering the context helps the model to correctly recognize characters individually, training with such a synthetic dataset interferes the model with learning the contextual information (i.e., relation among characters), which is also important for predicting the whole word. Based on this motivation, we propose a novel Context-Aware and Free Experts Network (CAFE-Net) using two experts: 1) context-aware expert learns the contextual representation trained with a long-tailed dataset composed of common words used in everyday life and 2) context-free expert focuses on correctly predicting individual characters by utilizing a dataset with a balanced number of characters. By training two experts to focus on learning contextual and visual representations, respectively, we propose a novel confidence ensemble method to compensate the limitation of each expert. Through the experiments, we demonstrate that CAFE-Net improves the STR performance on languages containing numerous number of characters. Moreover, we show that CAFE-Net is easily applicable to various STR models.