On the Challenges of Physical Implementations of RBMs

TL;DR

This paper investigates the potential and limitations of using physical systems to implement RBMs, aiming to reduce sampling costs but facing hardware constraints, through simulations inspired by quantum computing hardware.

Contribution

It provides a systematic simulation-based analysis of hardware limitations on physical RBM implementations, highlighting their impact on sampling quality.

Findings

Hardware constraints significantly affect sampling accuracy.

Physical implementations can bypass some MCMC costs.

Limitations include low precision and restricted topology.

Abstract

Restricted Boltzmann machines (RBMs) are powerful machine learning models, but learning and some kinds of inference in the model require sampling-based approximations, which, in classical digital computers, are implemented using expensive MCMC. Physical computation offers the opportunity to reduce the cost of sampling by building physical systems whose natural dynamics correspond to drawing samples from the desired RBM distribution. Such a system avoids the burn-in and mixing cost of a Markov chain. However, hardware implementations of this variety usually entail limitations such as low-precision and limited range of the parameters and restrictions on the size and topology of the RBM. We conduct software simulations to determine how harmful each of these restrictions is. Our simulations are designed to reproduce aspects of the D-Wave quantum computer, but the issues we investigate arise in most forms of physical computation.