Progressively Exploring and Exploiting Inference Data to Break Fine-Grained Classification Barrier

TL;DR

This paper introduces a novel inference-time learning paradigm and the EXP2 method to improve fine-grained classification by leveraging cost-free inference data, addressing real-world data diversity and dynamic semantics.

Contribution

It proposes a new paradigm for progressive learning during inference and the EXP2 strategy to utilize inference data for better fine-grained classification.

Findings

Effective in real-world scenarios with diverse and changing data

Improves classification accuracy by leveraging inference data

Guides future research in fine-grained classification

Abstract

Current fine-grained classification research primarily focuses on fine-grained feature learning. However, in real-world scenarios, fine-grained data annotation is challenging, and the features and semantics are highly diverse and frequently changing. These issues create inherent barriers between traditional experimental settings and real-world applications, limiting the effectiveness of conventional fine-grained classification methods. Although some recent studies have provided potential solutions to these issues, most of them still rely on limited supervised information and thus fail to offer effective solutions. In this paper, based on theoretical analysis, we propose a novel learning paradigm to break the barriers in fine-grained classification. This paradigm enables the model to progressively learn during inference, thereby leveraging cost-free data at inference time to more accurately represent fine-grained categories and adapt to dynamic semantic changes. On this basis, an efficient EXPloring and EXPloiting strategy and method (EXP2) is designed. Thereinto, useful inference data samples are explored according to class representations and exploited to optimize classifiers. Experimental results demonstrate the general effectiveness of our method, providing guidance for future in-depth understanding and exploration of real-world fine-grained classification.