Semi-Supervised Learning in the Few-Shot Zero-Shot Scenario

TL;DR

This paper extends semi-supervised learning to scenarios with missing classes in labeled data by introducing a KL-divergence penalty, significantly improving accuracy in few-shot, zero-shot settings on image classification benchmarks.

Contribution

It proposes a novel augmentation to existing SSL methods that effectively handles missing classes by penalizing class frequency divergence.

Findings

Significant accuracy improvements over state-of-the-art SSL methods.

Effective in few-shot and zero-shot learning scenarios.

Validated on CIFAR-100 and STL-10 datasets.

Abstract

Semi-Supervised Learning (SSL) is a framework that utilizes both labeled and unlabeled data to enhance model performance. Conventional SSL methods operate under the assumption that labeled and unlabeled data share the same label space. However, in practical real-world scenarios, especially when the labeled training dataset is limited in size, some classes may be totally absent from the labeled set. To address this broader context, we propose a general approach to augment existing SSL methods, enabling them to effectively handle situations where certain classes are missing. This is achieved by introducing an additional term into their objective function, which penalizes the KL-divergence between the probability vectors of the true class frequencies and the inferred class frequencies. Our experimental results reveal significant improvements in accuracy when compared to state-of-the-art SSL, open-set SSL, and open-world SSL methods. We conducted these experiments on two benchmark image classification datasets, CIFAR-100 and STL-10, with the most remarkable improvements observed when the labeled data is severely limited, with only a few labeled examples per class