Unlocking the Power of Boltzmann Machines by Parallelizable Sampler and Efficient Temperature Estimation

TL;DR

This paper introduces a parallelizable Boltzmann sampler inspired by quantum optimization and an efficient temperature estimation method, enabling scalable learning of more expressive Boltzmann machines.

Contribution

It proposes Langevin SB for parallel sampling and CEM for temperature estimation, forming a new scalable learning framework for general Boltzmann machines.

Findings

LSB achieves parallel sampling with accuracy comparable to MCMC.

CEM effectively estimates inverse temperature during learning.

The combined SAL framework enhances the expressiveness and efficiency of Boltzmann machine training.

Abstract

Boltzmann machines (BMs) are powerful energy-based generative models, but their heavy training cost has largely confined practical use to Restricted BMs (RBMs) trained with an efficient learning method called contrastive divergence. More accurate learning typically requires Markov chain Monte Carlo (MCMC) Boltzmann sampling, but it is time-consuming due to the difficulty of parallelization for more expressive models. To address this limitation, we first propose a new Boltzmann sampler inspired by a quantum-inspired combinatorial optimization called simulated bifurcation (SB). This SB-inspired approach, which we name Langevin SB (LSB), enables parallelized sampling while maintaining accuracy comparable to MCMC. Furthermore, this is applicable not only to RBMs but also to BMs with general couplings. However, LSB cannot control the inverse temperature of the output Boltzmann distribution, which hinders learning and degrades performance. To overcome this limitation, we also developed an efficient method for estimating the inverse temperature during the learning process, which we call conditional expectation matching (CEM). By combining LSB and CEM, we establish an efficient learning framework for BMs with greater expressive power than RBMs. We refer to this framework as sampler-adaptive learning (SAL). SAL opens new avenues for energy-based generative modeling beyond RBMs.