Exploring Beyond Logits: Hierarchical Dynamic Labeling Based on Embeddings for Semi-Supervised Classification

TL;DR

This paper introduces a Hierarchical Dynamic Labeling (HDL) algorithm that leverages image embeddings for pseudo-labeling in semi-supervised learning, improving robustness and adaptability over traditional confidence-based methods.

Contribution

The paper proposes a novel HDL method that does not rely on model predictions and introduces an adaptive hyperparameter selection, enhancing semi-supervised classification performance.

Findings

HDL improves classification accuracy across datasets.

Embedding-based labeling outperforms confidence-based pseudo-labeling.

The approach is compatible with general image encoders like CLIP.

Abstract

In semi-supervised learning, methods that rely on confidence learning to generate pseudo-labels have been widely proposed. However, increasing research finds that when faced with noisy and biased data, the model's representation network is more reliable than the classification network. Additionally, label generation methods based on model predictions often show poor adaptability across different datasets, necessitating customization of the classification network. Therefore, we propose a Hierarchical Dynamic Labeling (HDL) algorithm that does not depend on model predictions and utilizes image embeddings to generate sample labels. We also introduce an adaptive method for selecting hyperparameters in HDL, enhancing its versatility. Moreover, HDL can be combined with general image encoders (e.g., CLIP) to serve as a fundamental data processing module. We extract embeddings from datasets with class-balanced and long-tailed distributions using pre-trained semi-supervised models. Subsequently, samples are re-labeled using HDL, and the re-labeled samples are used to further train the semi-supervised models. Experiments demonstrate improved model performance, validating the motivation that representation networks are more reliable than classifiers or predictors. Our approach has the potential to change the paradigm of pseudo-label generation in semi-supervised learning.