# Rethink Long-tailed Recognition with Vision Transformers

## TL;DR

This paper explores the challenges of long-tailed recognition using Vision Transformers, proposing unsupervised learning and a new calibration metric to improve understanding and performance on imbalanced datasets.

## Contribution

It introduces the Predictive Distribution Calibration (PDC) metric and demonstrates how unsupervised learning can better utilize long-tailed data with Vision Transformers.

## Key findings

- ViT is difficult to train with long-tailed data
- Unsupervised learning helps ViT learn generalized features
- PDC effectively measures model calibration and predictive preferences

## Abstract

In the real world, data tends to follow long-tailed distributions w.r.t. class or attribution, motivating the challenging Long-Tailed Recognition (LTR) problem. In this paper, we revisit recent LTR methods with promising Vision Transformers (ViT). We figure out that 1) ViT is hard to train with long-tailed data. 2) ViT learns generalized features in an unsupervised manner, like mask generative training, either on long-tailed or balanced datasets. Hence, we propose to adopt unsupervised learning to utilize long-tailed data. Furthermore, we propose the Predictive Distribution Calibration (PDC) as a novel metric for LTR, where the model tends to simply classify inputs into common classes. Our PDC can measure the model calibration of predictive preferences quantitatively. On this basis, we find many LTR approaches alleviate it slightly, despite the accuracy improvement. Extensive experiments on benchmark datasets validate that PDC reflects the model's predictive preference precisely, which is consistent with the visualization.