Restricted Boltzmann Machine with Multivalued Hidden Variables: a model suppressing over-fitting

TL;DR

This paper introduces a modified restricted Boltzmann machine with multivalued hidden variables, aiming to improve generalization and reduce over-fitting, validated through experiments on artificial and MNIST data.

Contribution

The paper proposes a simple extension to RBMs with multivalued hidden variables to enhance generalization and suppress over-fitting.

Findings

The proposed model outperforms conventional RBMs in experiments.

Numerical experiments show improved generalization.

Effective in classification tasks with MNIST.

Abstract

Generalization is one of the most important issues in machine learning problems. In this study, we consider generalization in restricted Boltzmann machines (RBMs). We propose an RBM with multivalued hidden variables, which is a simple extension of conventional RBMs. We demonstrate that the proposed model is better than the conventional model via numerical experiments for contrastive divergence learning with artificial data and a classification problem with MNIST.