Imagine by Reasoning: A Reasoning-Based Implicit Semantic Data Augmentation for Long-Tailed Classification

TL;DR

This paper introduces a reasoning-based data augmentation method that enhances tail class samples in long-tailed classification by borrowing transformation directions from similar categories using covariance matrices and a knowledge graph.

Contribution

It proposes a novel implicit semantic augmentation approach leveraging covariance matrices and a knowledge graph to improve long-tailed classification performance.

Findings

Significant improvement on CIFAR-100-LT, ImageNet-LT, and iNaturalist 2018 datasets.

Outperforms state-of-the-art methods in long-tailed classification.

Effective tail sample enhancement through category relation propagation.

Abstract

Real-world data often follows a long-tailed distribution, which makes the performance of existing classification algorithms degrade heavily. A key issue is that samples in tail categories fail to depict their intra-class diversity. Humans can imagine a sample in new poses, scenes, and view angles with their prior knowledge even if it is the first time to see this category. Inspired by this, we propose a novel reasoning-based implicit semantic data augmentation method to borrow transformation directions from other classes. Since the covariance matrix of each category represents the feature transformation directions, we can sample new directions from similar categories to generate definitely different instances. Specifically, the long-tailed distributed data is first adopted to train a backbone and a classifier. Then, a covariance matrix for each category is estimated, and a knowledge graph is constructed to store the relations of any two categories. Finally, tail samples are adaptively enhanced via propagating information from all the similar categories in the knowledge graph. Experimental results on CIFAR-100-LT, ImageNet-LT, and iNaturalist 2018 have demonstrated the effectiveness of our proposed method compared with the state-of-the-art methods.