Monte Carlo methods for massively parallel computers

TL;DR

This paper discusses the opportunities, challenges, and design principles of implementing Monte Carlo simulations on massively parallel computers like GPUs and FPGAs, focusing on statistical physics applications.

Contribution

It provides an overview of parallel Monte Carlo methods, addressing algorithmic challenges and offering design principles for efficient implementation on modern hardware.

Findings

Parallel Monte Carlo methods enable simulations of complex physical systems.

Random number generation is a key challenge in parallel Monte Carlo.

Design principles improve performance and scalability of parallel Monte Carlo codes.

Abstract

Applications that require substantial computational resources today cannot avoid the use of heavily parallel machines. Embracing the opportunities of parallel computing and especially the possibilities provided by a new generation of massively parallel accelerator devices such as GPUs, Intel's Xeon Phi or even FPGAs enables applications and studies that are inaccessible to serial programs. Here we outline the opportunities and challenges of massively parallel computing for Monte Carlo simulations in statistical physics, with a focus on the simulation of systems exhibiting phase transitions and critical phenomena. This covers a range of canonical ensemble Markov chain techniques as well as generalized ensembles such as multicanonical simulations and population annealing. While the examples discussed are for simulations of spin systems, many of the methods are more general and moderate modifications allow them to be applied to other lattice and off-lattice problems including polymers and particle systems. We discuss important algorithmic requirements for such highly parallel simulations, such as the challenges of random-number generation for such cases, and outline a number of general design principles for parallel Monte Carlo codes to perform well.