Transfer of Pretrained Model Weights Substantially Improves Semi-Supervised Image Classification

TL;DR

This paper demonstrates that transfer learning with pretrained weights significantly enhances semi-supervised image classification accuracy, especially with limited labeled data, across multiple datasets.

Contribution

It provides an empirical evaluation showing transfer learning consistently improves semi-supervised learning performance regardless of loss functions used.

Findings

Transfer learning substantially boosts accuracy with few labeled examples.

Pretrained weights from Imagenet improve results across datasets.

The improvement is consistent across different similarity metrics and label propagation methods.

Abstract

Deep neural networks produce state-of-the-art results when trained on a large number of labeled examples but tend to overfit when small amounts of labeled examples are used for training. Creating a large number of labeled examples requires considerable resources, time, and effort. If labeling new data is not feasible, so-called semi-supervised learning can achieve better generalisation than purely supervised learning by employing unlabeled instances as well as labeled ones. The work presented in this paper is motivated by the observation that transfer learning provides the opportunity to potentially further improve performance by exploiting models pretrained on a similar domain. More specifically, we explore the use of transfer learning when performing semi-supervised learning using self-learning. The main contribution is an empirical evaluation of transfer learning using different combinations of similarity metric learning methods and label propagation algorithms in semi-supervised learning. We find that transfer learning always substantially improves the model's accuracy when few labeled examples are available, regardless of the type of loss used for training the neural network. This finding is obtained by performing extensive experiments on the SVHN, CIFAR10, and Plant Village image classification datasets and applying pretrained weights from Imagenet for transfer learning.