Regularization With Stochastic Transformations and Perturbations for Deep Semi-Supervised Learning

TL;DR

This paper introduces a semi-supervised learning method for convolutional neural networks that leverages stochastic data transformations and an unsupervised loss to improve generalization using limited labeled data.

Contribution

It proposes an unsupervised loss function that exploits the stochasticity of data augmentation and dropout to enhance semi-supervised CNN training.

Findings

Improved accuracy on benchmark datasets.

Enhanced model stability and generalization.

Effective use of unlabeled data through stochastic consistency.

Abstract

Effective convolutional neural networks are trained on large sets of labeled data. However, creating large labeled datasets is a very costly and time-consuming task. Semi-supervised learning uses unlabeled data to train a model with higher accuracy when there is a limited set of labeled data available. In this paper, we consider the problem of semi-supervised learning with convolutional neural networks. Techniques such as randomized data augmentation, dropout and random max-pooling provide better generalization and stability for classifiers that are trained using gradient descent. Multiple passes of an individual sample through the network might lead to different predictions due to the non-deterministic behavior of these techniques. We propose an unsupervised loss function that takes advantage of the stochastic nature of these methods and minimizes the difference between the predictions of multiple passes of a training sample through the network. We evaluate the proposed method on several benchmark datasets.