Investigating the Effect of Intraclass Variability in Temporal Ensembling

TL;DR

This study investigates how intraclass variability affects the performance of temporal ensembling in semi-supervised learning, highlighting the importance of seed size and seed type on accuracy.

Contribution

It provides a preliminary analysis of intraclass variability's impact on temporal ensembling, emphasizing seed selection and dataset characteristics.

Findings

Higher intraclass variability reduces accuracy

More seed images improve performance

Seed type influences efficiency and accuracy

Abstract

Temporal Ensembling is a semi-supervised approach that allows training deep neural network models with a small number of labeled images. In this paper, we present our preliminary study on the effect of intraclass variability on temporal ensembling, with a focus on seed size and seed type, respectively. Through our experiments we find that (a) there is a significant drop in accuracy with datasets that offer high intraclass variability, (b) more seed images offer consistently higher accuracy across the datasets, and (c) seed type indeed has an impact on the overall efficiency, where it produces a spectrum of accuracy both lower and higher. Additionally, based on our experiments, we also find KMNIST to be a competitive baseline for temporal ensembling.