Classification-Specific Parts for Improving Fine-Grained Visual Categorization

TL;DR

This paper introduces a novel method for fine-grained visual categorization that automatically estimates and selects relevant image parts using gradient-based importance, improving classification accuracy.

Contribution

It proposes a classification-specific part estimation technique that automatically determines relevant regions with spatial extent, surpassing point-based annotations.

Findings

Effective part selection improves classification accuracy

Method outperforms existing approaches on standard datasets

Automatically determines spatial extent of parts

Abstract

Fine-grained visual categorization is a classification task for distinguishing categories with high intra-class and small inter-class variance. While global approaches aim at using the whole image for performing the classification, part-based solutions gather additional local information in terms of attentions or parts. We propose a novel classification-specific part estimation that uses an initial prediction as well as back-propagation of feature importance via gradient computations in order to estimate relevant image regions. The subsequently detected parts are then not only selected by a-posteriori classification knowledge, but also have an intrinsic spatial extent that is determined automatically. This is in contrast to most part-based approaches and even to available ground-truth part annotations, which only provide point coordinates and no additional scale information. We show in our experiments on various widely-used fine-grained datasets the effectiveness of the mentioned part selection method in conjunction with the extracted part features.