Smooth Pseudo-Labeling
Nikolaos Karaliolios, Herv\'e Le Borgne, Florian Chabot

TL;DR
This paper introduces a Smooth Pseudo-Labeling loss function to improve semi-supervised learning stability and performance, especially with scarce labels, by smoothing derivative discontinuities in pseudo-labeling methods.
Contribution
It proposes a novel smoothing technique for pseudo-labeling loss functions, enhancing stability and performance in semi-supervised learning without extra modules or hyperparameters.
Findings
Significant performance improvement with scarce labels
Enhanced robustness to hyperparameter variations
Better results on a new random-label selection benchmark
Abstract
Semi-Supervised Learning (SSL) seeks to leverage large amounts of non-annotated data along with the smallest amount possible of annotated data in order to achieve the same level of performance as if all data were annotated. A fruitful method in SSL is Pseudo-Labeling (PL), which, however, suffers from the important drawback that the associated loss function has discontinuities in its derivatives, which cause instabilities in performance when labels are very scarce. In the present work, we address this drawback with the introduction of a Smooth Pseudo-Labeling (SP L) loss function. It consists in adding a multiplicative factor in the loss function that smooths out the discontinuities in the derivative due to thresholding. In our experiments, we test our improvements on FixMatch and show that it significantly improves the performance in the regime of scarce labels, without addition of any…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAmino Acid Enzymes and Metabolism · Pharmacy and Medical Practices
MethodsFixMatch
