Multi-layered Discriminative Restricted Boltzmann Machine with Untrained Probabilistic Layer

TL;DR

This paper introduces a multi-layered probabilistic neural network combining a discriminative restricted Boltzmann machine with an untrained probabilistic layer inspired by extreme learning machines, enhancing noise robustness and generalization.

Contribution

The novel MDRBM model integrates an untrained probabilistic layer with DRBM, improving noise immunity and generalization in classification tasks.

Findings

MDRBM outperforms existing models on benchmark datasets.

Enhanced noise robustness demonstrated in experiments.

Model achieves superior generalization capabilities.

Abstract

An extreme learning machine (ELM) is a three-layered feed-forward neural network having untrained parameters, which are randomly determined before training. Inspired by the idea of ELM, a probabilistic untrained layer called a probabilistic-ELM (PELM) layer is proposed, and it is combined with a discriminative restricted Boltzmann machine (DRBM), which is a probabilistic three-layered neural network for solving classification problems. The proposed model is obtained by stacking DRBM on the PELM layer. The resultant model (i.e., multi-layered DRBM (MDRBM)) forms a probabilistic four-layered neural network. In MDRBM, the parameters in the PELM layer can be determined using Gaussian-Bernoulli restricted Boltzmann machine. Owing to the PELM layer, MDRBM obtains a strong immunity against noise in inputs, which is one of the most important advantages of MDRBM. Numerical experiments using some benchmark datasets, MNIST, Fashion-MNIST, Urban Land Cover, and CIFAR-10, demonstrate that MDRBM is superior to other existing models, particularly, in terms of the noise-robustness property (or, in other words, the generalization property).