Learning Boltzmann Machine with EM-like Method

TL;DR

This paper introduces an EM-like algorithm for training Boltzmann machines with unconstrained connectivity, utilizing Monte Carlo methods and approximations to improve efficiency and performance assessment.

Contribution

It presents a novel EM-like training method for Boltzmann machines, connecting it to contrastive divergence and proposing a new performance measure.

Findings

EM-like method matches contrastive divergence in restricted Boltzmann machines

Proposed performance measure has complexity O(Rmn)

Numerical experiments demonstrate effectiveness of the method

Abstract

We propose an expectation-maximization-like(EMlike) method to train Boltzmann machine with unconstrained connectivity. It adopts Monte Carlo approximation in the E-step, and replaces the intractable likelihood objective with efficiently computed objectives or directly approximates the gradient of likelihood objective in the M-step. The EM-like method is a modification of alternating minimization. We prove that EM-like method will be the exactly same with contrastive divergence in restricted Boltzmann machine if the M-step of this method adopts special approximation. We also propose a new measure to assess the performance of Boltzmann machine as generative models of data, and its computational complexity is O(Rmn). Finally, we demonstrate the performance of EM-like method using numerical experiments.