Rethinking Guidance Information to Utilize Unlabeled Samples:A Label Encoding Perspective

TL;DR

This paper introduces Label-Encoding Risk Minimization (LERM), a novel approach that improves learning from unlabeled data by ensuring both discriminability and diversity of predictions, outperforming existing methods in label-scarce scenarios.

Contribution

LERM is a new method that estimates label encodings from unlabeled data and aligns them with true labels, enhancing learning robustness with limited labels.

Findings

LERM outperforms existing methods in label-insufficient scenarios.

Theoretical analysis links LERM with ERM and EntMin.

LERM effectively balances prediction discriminability and diversity.

Abstract

Empirical Risk Minimization (ERM) is fragile in scenarios with insufficient labeled samples. A vanilla extension of ERM to unlabeled samples is Entropy Minimization (EntMin), which employs the soft-labels of unlabeled samples to guide their learning. However, EntMin emphasizes prediction discriminability while neglecting prediction diversity. To alleviate this issue, in this paper, we rethink the guidance information to utilize unlabeled samples. By analyzing the learning objective of ERM, we find that the guidance information for labeled samples in a specific category is the corresponding label encoding. Inspired by this finding, we propose a Label-Encoding Risk Minimization (LERM). It first estimates the label encodings through prediction means of unlabeled samples and then aligns them with their corresponding ground-truth label encodings. As a result, the LERM ensures both prediction discriminability and diversity, and it can be integrated into existing methods as a plugin. Theoretically, we analyze the relationships between LERM and ERM as well as EntMin. Empirically, we verify the superiority of the LERM under several label insufficient scenarios. The codes are available at https://github.com/zhangyl660/LERM.