Interpolation-based Contrastive Learning for Few-Label Semi-Supervised Learning

TL;DR

This paper introduces an interpolation-based contrastive learning method for semi-supervised learning that significantly improves performance in extremely low-label scenarios by constructing reliable positive pairs and guiding embedding changes linearly.

Contribution

The paper proposes a novel interpolation-based contrastive loss that enhances semi-supervised learning with very limited labels, addressing issues of semantic drift in data augmentation.

Findings

Outperforms existing methods by 5.3% on CIFAR-10 with 2 labels per class.

Achieves 88.73% accuracy with only two labels per class.

Improves state-of-the-art algorithms using the proposed strategy.

Abstract

Semi-supervised learning (SSL) has long been proved to be an effective technique to construct powerful models with limited labels. In the existing literature, consistency regularization-based methods, which force the perturbed samples to have similar predictions with the original ones have attracted much attention for their promising accuracy. However, we observe that, the performance of such methods decreases drastically when the labels get extremely limited, e.g., 2 or 3 labels for each category. Our empirical study finds that the main problem lies with the drifting of semantic information in the procedure of data augmentation. The problem can be alleviated when enough supervision is provided. However, when little guidance is available, the incorrect regularization would mislead the network and undermine the performance of the algorithm. To tackle the problem, we (1) propose an interpolation-based method to construct more reliable positive sample pairs; (2) design a novel contrastive loss to guide the embedding of the learned network to change linearly between samples so as to improve the discriminative capability of the network by enlarging the margin decision boundaries. Since no destructive regularization is introduced, the performance of our proposed algorithm is largely improved. Specifically, the proposed algorithm outperforms the second best algorithm (Comatch) with 5.3% by achieving 88.73% classification accuracy when only two labels are available for each class on the CIFAR-10 dataset. Moreover, we further prove the generality of the proposed method by improving the performance of the existing state-of-the-art algorithms considerably with our proposed strategy.