Semi-Supervised Learning guided by the Generalized Bayes Rule under Soft Revision

TL;DR

This paper introduces a robust semi-supervised learning approach using the Gamma-Maximin method with soft revision, leveraging credal sets of priors to improve pseudo-label selection, especially with limited labeled data.

Contribution

It formalizes the Gamma-Maximin method with soft revision for pseudo-label selection and demonstrates its effectiveness with logistic models in semi-supervised learning.

Findings

Achieves promising results with low labeled data proportions

Provides a formal optimization framework for pseudo-label selection

Shows improved robustness over traditional methods

Abstract

We provide a theoretical and computational investigation of the Gamma-Maximin method with soft revision, which was recently proposed as a robust criterion for pseudo-label selection (PLS) in semi-supervised learning. Opposed to traditional methods for PLS we use credal sets of priors ("generalized Bayes") to represent the epistemic modeling uncertainty. These latter are then updated by the Gamma-Maximin method with soft revision. We eventually select pseudo-labeled data that are most likely in light of the least favorable distribution from the so updated credal set. We formalize the task of finding optimal pseudo-labeled data w.r.t. the Gamma-Maximin method with soft revision as an optimization problem. A concrete implementation for the class of logistic models then allows us to compare the predictive power of the method with competing approaches. It is observed that the Gamma-Maximin method with soft revision can achieve very promising results, especially when the proportion of labeled data is low.