Extremely Fine-Grained Visual Classification over Resembling Glyphs in the Wild

TL;DR

This paper introduces challenging datasets and a novel two-stage contrastive learning approach for extremely fine-grained visual classification of similar-looking glyphs in natural scenes, improving recognition accuracy.

Contribution

It presents the first benchmark datasets for resembling glyphs and a new two-stage contrastive learning method combining classification and contrastive learning in Euclidean and Angular spaces.

Findings

Our approach outperforms state-of-the-art fine-grained classification methods.

Contrastive learning enhances feature representation for similar glyphs.

The method is effective with both CNN and Transformer backbones.

Abstract

Text recognition in the wild is an important technique for digital maps and urban scene understanding, in which the natural resembling properties between glyphs is one of the major reasons that lead to wrong recognition results. To address this challenge, we introduce two extremely fine-grained visual recognition benchmark datasets that contain very challenging resembling glyphs (characters/letters) in the wild to be distinguished. Moreover, we propose a simple yet effective two-stage contrastive learning approach to the extremely fine-grained recognition task of resembling glyphs discrimination. In the first stage, we utilize supervised contrastive learning to leverage label information to warm-up the backbone network. In the second stage, we introduce CCFG-Net, a network architecture that integrates classification and contrastive learning in both Euclidean and Angular spaces, in which contrastive learning is applied in both supervised learning and pairwise discrimination manners to enhance the model's feature representation capability. Overall, our proposed approach effectively exploits the complementary strengths of contrastive learning and classification, leading to improved recognition performance on the resembling glyphs. Comparative evaluations with state-of-the-art fine-grained classification approaches under both Convolutional Neural Network (CNN) and Transformer backbones demonstrate the superiority of our proposed method.