SimPLE: Similar Pseudo Label Exploitation for Semi-Supervised Classification

TL;DR

SimPLE introduces a novel semi-supervised classification method that leverages high-confidence similar unlabeled data through a Pair Loss, improving performance across multiple datasets and transfer learning scenarios.

Contribution

The paper proposes a new Pair Loss for semi-supervised learning that exploits similarities among high-confidence unlabeled data, enhancing existing techniques like MixMatch.

Findings

Significant performance improvements on CIFAR-100 and Mini-ImageNet.

Competitive results on CIFAR-10 and SVHN.

Outperforms state-of-the-art in transfer learning settings.

Abstract

A common classification task situation is where one has a large amount of data available for training, but only a small portion is annotated with class labels. The goal of semi-supervised training, in this context, is to improve classification accuracy by leverage information not only from labeled data but also from a large amount of unlabeled data. Recent works have developed significant improvements by exploring the consistency constrain between differently augmented labeled and unlabeled data. Following this path, we propose a novel unsupervised objective that focuses on the less studied relationship between the high confidence unlabeled data that are similar to each other. The new proposed Pair Loss minimizes the statistical distance between high confidence pseudo labels with similarity above a certain threshold. Combining the Pair Loss with the techniques developed by the MixMatch family, our proposed SimPLE algorithm shows significant performance gains over previous algorithms on CIFAR-100 and Mini-ImageNet, and is on par with the state-of-the-art methods on CIFAR-10 and SVHN. Furthermore, SimPLE also outperforms the state-of-the-art methods in the transfer learning setting, where models are initialized by the weights pre-trained on ImageNet or DomainNet-Real. The code is available at github.com/zijian-hu/SimPLE.