Semi-Supervised Learning with Ladder Networks

TL;DR

This paper introduces a semi-supervised deep learning model that combines supervised and unsupervised training, extending the Ladder network to achieve state-of-the-art results on MNIST and CIFAR-10 datasets.

Contribution

It extends the Ladder network by integrating supervision, enabling effective semi-supervised learning without layer-wise pre-training.

Findings

Achieves state-of-the-art semi-supervised classification accuracy on MNIST and CIFAR-10.

Demonstrates effective learning with limited labeled data.

Outperforms previous semi-supervised methods.

Abstract

We combine supervised learning with unsupervised learning in deep neural networks. The proposed model is trained to simultaneously minimize the sum of supervised and unsupervised cost functions by backpropagation, avoiding the need for layer-wise pre-training. Our work builds on the Ladder network proposed by Valpola (2015), which we extend by combining the model with supervision. We show that the resulting model reaches state-of-the-art performance in semi-supervised MNIST and CIFAR-10 classification, in addition to permutation-invariant MNIST classification with all labels.