Policy-guided Monte Carlo on general state spaces: Application to glass-forming mixtures

TL;DR

This paper extends policy-guided Monte Carlo methods to general state spaces and applies them to glass-forming mixtures, demonstrating significant efficiency improvements in some models but limited gains in others.

Contribution

The work generalizes policy-guided Monte Carlo to mixed discrete and continuous spaces and evaluates its performance on glass-forming models.

Findings

Two orders of magnitude speed-up for additive soft sphere mixture

Limited speed-up for Kob-Andersen model

Discussion of current limitations and future improvements

Abstract

Policy-guided Monte Carlo is an adaptive method to simulate classical interacting systems. It adjusts the proposal distribution of the Metropolis-Hastings algorithm to maximize the sampling efficiency, using a formalism inspired by reinforcement learning. In this work, we first extend the policy-guided method to deal with a general state space, comprising, for instance, both discrete and continuous degrees of freedom, and then apply it to a few paradigmatic models of glass-forming mixtures. We assess the efficiency of a set of physically inspired moves whose proposal distributions are optimized through on-policy learning. Compared to conventional Monte Carlo methods, the optimized proposals are two orders of magnitude faster for an additive soft sphere mixture but yield a much more limited speed-up for the well-studied Kob-Andersen model. We discuss the current limitations of the method and suggest possible ways to improve it.