Part-guided Relational Transformers for Fine-grained Visual Recognition

TL;DR

This paper introduces PART, a unified framework using relational transformers for fine-grained visual recognition, which automatically discovers discriminative parts and models their relationships, achieving state-of-the-art results without extra inference complexity.

Contribution

The paper presents a novel part-guided relational transformer framework that automatically discovers discriminative regions and models their correlations for improved fine-grained recognition.

Findings

Achieves state-of-the-art performance on three benchmarks.

Effectively discovers discriminative parts without extra inference cost.

Enhances spatial interactions among semantic features.

Abstract

Fine-grained visual recognition is to classify objects with visually similar appearances into subcategories, which has made great progress with the development of deep CNNs. However, handling subtle differences between different subcategories still remains a challenge. In this paper, we propose to solve this issue in one unified framework from two aspects, i.e., constructing feature-level interrelationships, and capturing part-level discriminative features. This framework, namely PArt-guided Relational Transformers (PART), is proposed to learn the discriminative part features with an automatic part discovery module, and to explore the intrinsic correlations with a feature transformation module by adapting the Transformer models from the field of natural language processing. The part discovery module efficiently discovers the discriminative regions which are highly-corresponded to the gradient descent procedure. Then the second feature transformation module builds correlations within the global embedding and multiple part embedding, enhancing spatial interactions among semantic pixels. Moreover, our proposed approach does not rely on additional part branches in the inference time and reaches state-of-the-art performance on 3 widely-used fine-grained object recognition benchmarks. Experimental results and explainable visualizations demonstrate the effectiveness of our proposed approach. The code can be found at https://github.com/iCVTEAM/PART.