Probability-density-aware Semi-supervised Learning

TL;DR

This paper introduces a probability-density-aware measure for semi-supervised learning that leverages cluster assumptions more effectively, leading to improved label propagation and better performance.

Contribution

It proposes a novel probability-density-aware measure and an associated label propagation algorithm, providing a theoretical foundation and demonstrating superior empirical results.

Findings

PMLP outperforms recent semi-supervised learning methods.

Pseudo-labeling is a special case of PMLP, explaining its effectiveness.

Theoretical analysis confirms the importance of density in SSL.

Abstract

Semi-supervised learning (SSL) assumes that neighbor points lie in the same category (neighbor assumption), and points in different clusters belong to various categories (cluster assumption). Existing methods usually rely on similarity measures to retrieve the similar neighbor points, ignoring cluster assumption, which may not utilize unlabeled information sufficiently and effectively. This paper first provides a systematical investigation into the significant role of probability density in SSL and lays a solid theoretical foundation for cluster assumption. To this end, we introduce a Probability-Density-Aware Measure (PM) to discern the similarity between neighbor points. To further improve Label Propagation, we also design a Probability-Density-Aware Measure Label Propagation (PMLP) algorithm to fully consider the cluster assumption in label propagation. Last but not least, we prove that traditional pseudo-labeling could be viewed as a particular case of PMLP, which provides a comprehensive theoretical understanding of PMLP's superior performance. Extensive experiments demonstrate that PMLP achieves outstanding performance compared with other recent methods.