Fast Ensemble Learning Using Adversarially-Generated Restricted Boltzmann Machines

TL;DR

This paper introduces a novel ensemble learning method that uses adversarially-generated Restricted Boltzmann Machines (RBMs) to improve image reconstruction and classification, reducing training complexity.

Contribution

It proposes generating RBMs via adversarial learning with pre-trained weights and combining them into ensembles as an alternative to extensive training.

Findings

Artificial RBM ensembles perform well in image tasks

The approach reduces training time for RBMs

Ensembles improve robustness against adversarial data

Abstract

Machine Learning has been applied in a wide range of tasks throughout the last years, ranging from image classification to autonomous driving and natural language processing. Restricted Boltzmann Machine (RBM) has received recent attention and relies on an energy-based structure to model data probability distributions. Notwithstanding, such a technique is susceptible to adversarial manipulation, i.e., slightly or profoundly modified data. An alternative to overcome the adversarial problem lies in the Generative Adversarial Networks (GAN), capable of modeling data distributions and generating adversarial data that resemble the original ones. Therefore, this work proposes to artificially generate RBMs using Adversarial Learning, where pre-trained weight matrices serve as the GAN inputs. Furthermore, it proposes to sample copious amounts of matrices and combine them into ensembles, alleviating the burden of training new models'. Experimental results demonstrate the suitability of the proposed approach under image reconstruction and image classification tasks, and describe how artificial-based ensembles are alternatives to pre-training vast amounts of RBMs.