Transfer and Share: Semi-Supervised Learning from Long-Tailed Data

TL;DR

This paper introduces TRAS, a semi-supervised learning method that effectively balances class distributions in long-tailed data, significantly improving minority class performance without complex optimization or data undersampling.

Contribution

TRAS transforms pseudo-label distributions to enhance minority class learning and merges models via shared features for simplicity and effectiveness.

Findings

TRAS outperforms state-of-the-art methods in accuracy.

Balanced pseudo-labels benefit minority-class training.

Shared feature learning improves overall and minority class performance.

Abstract

Long-Tailed Semi-Supervised Learning (LTSSL) aims to learn from class-imbalanced data where only a few samples are annotated. Existing solutions typically require substantial cost to solve complex optimization problems, or class-balanced undersampling which can result in information loss. In this paper, we present the TRAS (TRAnsfer and Share) to effectively utilize long-tailed semi-supervised data. TRAS transforms the imbalanced pseudo-label distribution of a traditional SSL model via a delicate function to enhance the supervisory signals for minority classes. It then transfers the distribution to a target model such that the minority class will receive significant attention. Interestingly, TRAS shows that more balanced pseudo-label distribution can substantially benefit minority-class training, instead of seeking to generate accurate pseudo-labels as in previous works. To simplify the approach, TRAS merges the training of the traditional SSL model and the target model into a single procedure by sharing the feature extractor, where both classifiers help improve the representation learning. According to extensive experiments, TRAS delivers much higher accuracy than state-of-the-art methods in the entire set of classes as well as minority classes.