Acceleration of Parallel Tempering for Markov Chain Monte Carlo methods

TL;DR

This paper presents a highly parallelized implementation of Parallel Tempering for Markov Chain Monte Carlo methods, significantly accelerating sampling in complex systems using CPU and GPU computing.

Contribution

It introduces a parallel implementation of Metropolis-Hastings with Parallel Tempering on CPUs and GPUs, achieving substantial speed-ups and providing a benchmark for future quantum algorithms.

Findings

52x speed-up with OpenMP on 48 CPU cores

986x speed-up with CUDA on GPUs

Provides a benchmark for quantum algorithm comparison

Abstract

Markov Chain Monte Carlo methods are algorithms used to sample probability distributions, commonly used to sample the Boltzmann distribution of physical/chemical models (e.g., protein folding, Ising model, etc.). This allows us to study their properties by sampling the most probable states of those systems. However, the sampling capabilities of these methods are not sufficiently accurate when handling complex configuration spaces. This has resulted in the development of new techniques that improve sampling accuracy, usually at the expense of increasing the computational cost. One of such techniques is Parallel Tempering which improves accuracy by running several replicas which periodically exchange their states. Computationally, this imposes a significant slow-down, which can be counteracted by means of parallelization. These schemes enable MCMC/PT techniques to be run more effectively and allow larger models to be studied. In this work, we present a parallel implementation of Metropolis-Hastings with Parallel Tempering, using OpenMP and CUDA for the parallelization in modern CPUs and GPUs, respectively. The results show a maximum speed-up of 52x using OpenMP with 48 cores, and of 986x speed-up with the CUDA version. Furthermore, the results serve as a basic benchmark to compare a future quantum implementation of the same algorithm.