Re-rank Coarse Classification with Local Region Enhanced Features for Fine-Grained Image Recognition

TL;DR

This paper introduces a retrieval-based coarse-to-fine framework that enhances fine-grained image recognition by re-ranking classification results using local region features and a weakly-supervised method for discriminative region detection.

Contribution

It proposes a novel re-ranking approach with local region features and a weakly-supervised region detection method for improved fine-grained recognition.

Findings

Achieves state-of-the-art results on CUB-200-2011, Stanford Cars, and FGVC Aircraft datasets.

Effectively integrates semantic global and discriminative local features.

Demonstrates the benefit of re-ranking TopN results for accuracy improvement.

Abstract

Fine-grained image recognition is very challenging due to the difficulty of capturing both semantic global features and discriminative local features. Meanwhile, these two features are not easy to be integrated, which are even conflicting when used simultaneously. In this paper, a retrieval-based coarse-to-fine framework is proposed, where we re-rank the TopN classification results by using the local region enhanced embedding features to improve the Top1 accuracy (based on the observation that the correct category usually resides in TopN results). To obtain the discriminative regions for distinguishing the fine-grained images, we introduce a weakly-supervised method to train a box generating branch with only image-level labels. In addition, to learn more effective semantic global features, we design a multi-level loss over an automatically constructed hierarchical category structure. Experimental results show that our method achieves state-of-the-art performance on three benchmarks: CUB-200-2011, Stanford Cars, and FGVC Aircraft. Also, visualizations and analysis are provided for better understanding.