Concept Drift and Long-Tailed Distribution in Fine-Grained Visual Categorization: Benchmark and Method

TL;DR

This paper introduces a new dataset capturing concept drift and long-tailed distribution in fine-grained visual categorization, along with a feature recombination method to improve learning under these conditions.

Contribution

It provides a real-world dataset reflecting concept drift and long-tailed distributions and proposes a novel feature recombination framework for better FGVC performance.

Findings

The dataset reveals significant concept drift and long-tail challenges in FGVC.

The proposed method improves classifier robustness in long-tailed, drifting scenarios.

Large vision-language models like CLIP show limitations under these conditions.

Abstract

Data is the foundation for the development of computer vision, and the establishment of datasets plays an important role in advancing the techniques of fine-grained visual categorization~(FGVC). In the existing FGVC datasets used in computer vision, it is generally assumed that each collected instance has fixed characteristics and the distribution of different categories is relatively balanced. In contrast, the real world scenario reveals the fact that the characteristics of instances tend to vary with time and exhibit a long-tailed distribution. Hence, the collected datasets may mislead the optimization of the fine-grained classifiers, resulting in unpleasant performance in real applications. Starting from the real-world conditions and to promote the practical progress of fine-grained visual categorization, we present a Concept Drift and Long-Tailed Distribution dataset. Specifically, the dataset is collected by gathering 11195 images of 250 instances in different species for 47 consecutive months in their natural contexts. The collection process involves dozens of crowd workers for photographing and domain experts for labeling. Meanwhile, we propose a feature recombination framework to address the learning challenges associated with CDLT. Experimental results validate the efficacy of our method while also highlighting the limitations of popular large vision-language models (e.g., CLIP) in the context of long-tailed distributions. This emphasizes the significance of CDLT as a benchmark for investigating these challenges.