Label-Aware Distribution Calibration for Long-tailed Classification

TL;DR

This paper introduces Label-Aware Distribution Calibration (LADC), a novel method that leverages knowledge from head classes to improve distribution estimation for tail classes in long-tailed classification, enhancing performance.

Contribution

LADC is the first approach to transfer statistics from head to tail classes for distribution calibration in long-tailed classification tasks.

Findings

LADC significantly outperforms existing methods on image and text datasets.

LADC provides more accurate distribution estimation as shown by visualization.

LADC improves classifier re-balancing for long-tailed data.

Abstract

Real-world data usually present long-tailed distributions. Training on imbalanced data tends to render neural networks perform well on head classes while much worse on tail classes. The severe sparseness of training instances for the tail classes is the main challenge, which results in biased distribution estimation during training. Plenty of efforts have been devoted to ameliorating the challenge, including data re-sampling and synthesizing new training instances for tail classes. However, no prior research has exploited the transferable knowledge from head classes to tail classes for calibrating the distribution of tail classes. In this paper, we suppose that tail classes can be enriched by similar head classes and propose a novel distribution calibration approach named as label-Aware Distribution Calibration LADC. LADC transfers the statistics from relevant head classes to infer the distribution of tail classes. Sampling from calibrated distribution further facilitates re-balancing the classifier. Experiments on both image and text long-tailed datasets demonstrate that LADC significantly outperforms existing methods.The visualization also shows that LADC provides a more accurate distribution estimation.