A brief survey on deep belief networks and introducing a new object oriented toolbox (DeeBNet)

TL;DR

This paper surveys deep belief networks and introduces a new object-oriented MATLAB toolbox that supports various sampling and sparsity methods, demonstrating its effectiveness on multiple datasets for feature learning and classification.

Contribution

It presents a novel, versatile MATLAB toolbox for DBNs with support for multiple sampling and sparsity techniques, compatible with Octave, and validated on diverse datasets.

Findings

The toolbox achieves classification errors comparable to state-of-the-art classifiers.

It can learn effective data representations from unlabeled data.

Supports GPU acceleration and multiple sampling methods.

Abstract

Nowadays, this is very popular to use the deep architectures in machine learning. Deep Belief Networks (DBNs) are deep architectures that use stack of Restricted Boltzmann Machines (RBM) to create a powerful generative model using training data. DBNs have many ability like feature extraction and classification that are used in many applications like image processing, speech processing and etc. This paper introduces a new object oriented MATLAB toolbox with most of abilities needed for the implementation of DBNs. In the new version, the toolbox can be used in Octave. According to the results of the experiments conducted on MNIST (image), ISOLET (speech), and 20 Newsgroups (text) datasets, it was shown that the toolbox can learn automatically a good representation of the input from unlabeled data with better discrimination between different classes. Also on all datasets, the obtained classification errors are comparable to those of state of the art classifiers. In addition, the toolbox supports different sampling methods (e.g. Gibbs, CD, PCD and our new FEPCD method), different sparsity methods (quadratic, rate distortion and our new normal method), different RBM types (generative and discriminative), using GPU, etc. The toolbox is a user-friendly open source software and is freely available on the website http://ceit.aut.ac.ir/~keyvanrad/DeeBNet%20Toolbox.html .