The Devil is in the Tails: Fine-grained Classification in the Wild

TL;DR

This paper investigates the challenges of fine-grained classification in long-tailed distributions, revealing that current methods excel with abundant data but struggle with rare categories, and highlighting the lack of transfer learning in current approaches.

Contribution

The study provides a detailed analysis of long-tailed distributions in fine-grained classification, emphasizing the need for new methods to handle data scarcity and transfer learning.

Findings

Peak performance on well-represented categories is high.

Classification degrades rapidly with fewer training examples.

Transfer learning is rarely used in current methods.

Abstract

The world is long-tailed. What does this mean for computer vision and visual recognition? The main two implications are (1) the number of categories we need to consider in applications can be very large, and (2) the number of training examples for most categories can be very small. Current visual recognition algorithms have achieved excellent classification accuracy. However, they require many training examples to reach peak performance, which suggests that long-tailed distributions will not be dealt with well. We analyze this question in the context of eBird, a large fine-grained classification dataset, and a state-of-the-art deep network classification algorithm. We find that (a) peak classification performance on well-represented categories is excellent, (b) given enough data, classification performance suffers only minimally from an increase in the number of classes, (c) classification performance decays precipitously as the number of training examples decreases, (d) surprisingly, transfer learning is virtually absent in current methods. Our findings suggest that our community should come to grips with the question of long tails.