Negative sampling in semi-supervised learning

TL;DR

This paper introduces NS3L, a negative sampling-based method that enhances semi-supervised learning algorithms like VAT and MixMatch, leading to significant performance improvements on standard benchmarks.

Contribution

The paper presents NS3L, a simple and effective negative sampling technique that can be integrated into existing SSL algorithms to improve their accuracy and robustness.

Findings

NS3L improves VAT performance significantly.

Adding NS3L to MixMatch yields better results.

Extensive experiments confirm the effectiveness of NS3L.

Abstract

We introduce Negative Sampling in Semi-Supervised Learning (NS3L), a simple, fast, easy to tune algorithm for semi-supervised learning (SSL). NS3L is motivated by the success of negative sampling/contrastive estimation. We demonstrate that adding the NS3L loss to state-of-the-art SSL algorithms, such as the Virtual Adversarial Training (VAT), significantly improves upon vanilla VAT and its variant, VAT with Entropy Minimization. By adding the NS3L loss to MixMatch, the current state-of-the-art approach on semi-supervised tasks, we observe significant improvements over vanilla MixMatch. We conduct extensive experiments on the CIFAR10, CIFAR100, SVHN and STL10 benchmark datasets.