Distribution Aligning Refinery of Pseudo-label for Imbalanced Semi-supervised Learning

TL;DR

This paper introduces DARP, a method that refines pseudo-labels in semi-supervised learning to address class imbalance issues, improving generalization to balanced test data.

Contribution

The paper proposes a convex optimization-based pseudo-label refinement method, DARP, which effectively mitigates class imbalance effects in semi-supervised learning.

Findings

DARP improves SSL performance on imbalanced datasets.

DARP is compatible with existing SSL methods.

DARP reduces bias towards majority classes.

Abstract

While semi-supervised learning (SSL) has proven to be a promising way for leveraging unlabeled data when labeled data is scarce, the existing SSL algorithms typically assume that training class distributions are balanced. However, these SSL algorithms trained under imbalanced class distributions can severely suffer when generalizing to a balanced testing criterion, since they utilize biased pseudo-labels of unlabeled data toward majority classes. To alleviate this issue, we formulate a convex optimization problem to softly refine the pseudo-labels generated from the biased model, and develop a simple algorithm, named Distribution Aligning Refinery of Pseudo-label (DARP) that solves it provably and efficiently. Under various class-imbalanced semi-supervised scenarios, we demonstrate the effectiveness of DARP and its compatibility with state-of-the-art SSL schemes.