Subspace Restricted Boltzmann Machine

TL;DR

The paper introduces the subspace Restricted Boltzmann Machine, a third-order model with multiplicative interactions, designed to capture pattern variations and improve digit recognition performance.

Contribution

It proposes a novel third-order Boltzmann machine with gate and subspace units, enhancing pattern variation modeling and feature pooling capabilities.

Findings

Effective in capturing pattern variations

Reduces reconstruction and classification errors

Demonstrates promising results on MNIST

Abstract

The subspace Restricted Boltzmann Machine (subspaceRBM) is a third-order Boltzmann machine where multiplicative interactions are between one visible and two hidden units. There are two kinds of hidden units, namely, gate units and subspace units. The subspace units reflect variations of a pattern in data and the gate unit is responsible for activating the subspace units. Additionally, the gate unit can be seen as a pooling feature. We evaluate the behavior of subspaceRBM through experiments with MNIST digit recognition task, measuring reconstruction error and classification error.