Fine-Grained Visual Classification via Simultaneously Learning of Multi-regional Multi-grained Features

TL;DR

This paper introduces a novel loss function, TDSA-Loss, that enables the simultaneous learning of multi-regional and multi-grained features for improved fine-grained visual classification, avoiding complex model structures.

Contribution

The paper proposes TDSA-Loss with multi-stage channel constraints and top-down spatial attention to effectively mine discriminative regions and details in fine-grained images.

Findings

Significant improvement on four fine-grained datasets.

Effective multi-regional and multi-grained feature extraction.

Modules outperform baseline methods in ablation studies.

Abstract

Fine-grained visual classification is a challenging task that recognizes the sub-classes belonging to the same meta-class. Large inter-class similarity and intra-class variance is the main challenge of this task. Most exiting methods try to solve this problem by designing complex model structures to explore more minute and discriminative regions. In this paper, we argue that mining multi-regional multi-grained features is precisely the key to this task. Specifically, we introduce a new loss function, termed top-down spatial attention loss (TDSA-Loss), which contains a multi-stage channel constrained module and a top-down spatial attention module. The multi-stage channel constrained module aims to make the feature channels in different stages category-aligned. Meanwhile, the top-down spatial attention module uses the attention map generated by high-level aligned feature channels to make middle-level aligned feature channels to focus on particular regions. Finally, we can obtain multiple discriminative regions on high-level feature channels and obtain multiple more minute regions within these discriminative regions on middle-level feature channels. In summary, we obtain multi-regional multi-grained features. Experimental results over four widely used fine-grained image classification datasets demonstrate the effectiveness of the proposed method. Ablative studies further show the superiority of two modules in the proposed method. Codes are available at: https://github.com/dongliangchang/Top-Down-Spatial-Attention-Loss.