SemiOccam: A Robust Semi-Supervised Image Recognition Network Using Sparse Labels

TL;DR

SemiOccam introduces a semi-supervised image recognition network that achieves high accuracy with minimal labeled data, using a hierarchical mixture density classification and mutual information optimization, while also addressing data leakage issues in STL-10.

Contribution

It proposes a novel hierarchical mixture density classification mechanism optimized via mutual information, achieving state-of-the-art results with minimal labeled data and a simple, efficient architecture.

Findings

Achieves over 95% accuracy with only 4 labeled samples per class.

Reduces training time to minutes with a simple architecture.

Identifies and removes data leakage in STL-10 dataset.

Abstract

We present SemiOccam, an image recognition network that leverages semi-supervised learning in a highly efficient manner. Existing works often rely on complex training techniques and architectures, requiring hundreds of GPU hours for training, while their generalization ability with extremely limited labeled data remains to be improved. To address these limitations, we construct a hierarchical mixture density classification mechanism by optimizing mutual information between feature representations and target classes, compressing redundant information while retaining crucial discriminative components. Experimental results demonstrate that our method achieves state-of-the-art performance on three commonly used datasets, with accuracy exceeding 95% on two of them using only 4 labeled samples per class, and its simple architecture keeps training time at the minute level. Notably, this paper reveals a long-overlooked data leakage issue in the STL-10 dataset for semi-supervised learning and removes duplicates to ensure reliable experimental results. We release the deduplicated CleanSTL-10 dataset to facilitate fair and reproducible research. Code available at https://github.com/Shu1L0n9/SemiOccam.