Probabilistic Compute-in-Memory Design For Efficient Markov Chain Monte Carlo Sampling

TL;DR

This paper introduces a novel compute-in-memory hardware design for Markov Chain Monte Carlo sampling, significantly improving energy efficiency and throughput by leveraging SRAM stochasticity, a pseudo-read operation, and new random number generation circuits.

Contribution

It proposes a new CIM-based MCMC architecture with innovative random number generation and data copying methods, achieving high efficiency and throughput improvements.

Findings

Energy efficiency of 0.53 pJ/sample

High throughput of 166.7 million samples per second

Energy efficiency improved by over 10^11 times compared to conventional processors

Abstract

Markov chain Monte Carlo (MCMC) is a widely used sampling method in modern artificial intelligence and probabilistic computing systems. It involves repetitive random number generations and thus often dominates the latency of probabilistic model computing. Hence, we propose a compute-in-memory (CIM) based MCMC design as a hardware acceleration solution. This work investigates SRAM bitcell stochasticity and proposes a novel ``pseudo-read'' operation, based on which we offer a block-wise random number generation circuit scheme for fast random number generation. Moreover, this work proposes a novel multi-stage exclusive-OR gate (MSXOR) design method to generate strictly uniformly distributed random numbers. The probability error deviating from a uniform distribution is suppressed under $10^{-5}$. Also, this work presents a novel in-memory copy circuit scheme to realize data copy inside a CIM sub-array, significantly reducing the use of R/W circuits for power saving. Evaluated in a commercial 28-nm process development kit, this CIM-based MCMC design generates 4-bit$\sim$32-bit samples with an energy efficiency of $0.53$~pJ/sample and high throughput of up to $166.7$M~samples/s. Compared to conventional processors, the overall energy efficiency improves $5.41\times10^{11}$ to $2.33\times10^{12}$ times.