Online Semi-Supervised Learning with Deep Hybrid Boltzmann Machines and Denoising Autoencoders

TL;DR

This paper introduces two new deep hybrid models, the Deep Hybrid Boltzmann Machine and Deep Hybrid Denoising Auto-encoder, designed for semi-supervised learning, especially in data-streams, demonstrating improved performance over existing methods.

Contribution

The paper presents novel deep hybrid architectures combining different models for semi-supervised learning with theoretical foundations and algorithms.

Findings

Improved predictive performance on discriminative tasks.

Effective handling of data-streams in lifelong learning scenarios.

Outperforms pseudo-labeled drop-out rectifier networks.

Abstract

Two novel deep hybrid architectures, the Deep Hybrid Boltzmann Machine and the Deep Hybrid Denoising Auto-encoder, are proposed for handling semi-supervised learning problems. The models combine experts that model relevant distributions at different levels of abstraction to improve overall predictive performance on discriminative tasks. Theoretical motivations and algorithms for joint learning for each are presented. We apply the new models to the domain of data-streams in work towards life-long learning. The proposed architectures show improved performance compared to a pseudo-labeled, drop-out rectifier network.