MetaDNS: Enhancing Exploration in Discrete Neural Samplers via Well-Tempered Metadynamics

TL;DR

MetaDNS introduces a novel framework that enhances exploration in discrete neural samplers by integrating well-tempered metadynamics, enabling better sampling of high-energy regions and accurate free energy estimation.

Contribution

It is the first to incorporate well-tempered metadynamics into discrete neural samplers, improving exploration and free energy reconstruction capabilities.

Findings

MetaDNS successfully reproduces thermodynamic distributions in challenging benchmarks.

It achieves comparable exploration to MCMC-based methods with fewer bias steps.

MetaDNS overcomes mode collapse and explores high-energy barriers effectively.

Abstract

Sampling from discrete distributions with multiple modes and energy barriers is fundamental to machine learning and computational physics. Recent discrete neural samplers like MDNS suffer from mode collapse and fail to sample high-energy barrier regions between modes, which is critical for free energy estimation and understanding phase transitions. We propose Metadynamics Discrete Neural Sampler (MetaDNS), a general framework integrating well-tempered metadynamics into discrete diffusion or autoregressive samplers. By maintaining an adaptive, history-dependent bias potential along selected low-dimensional coordinates, MetaDNS forces exploration of previously inaccessible regions, enabling free energy reconstruction infeasible with standard neural samplers due to a lack of high-energy samples. On challenging low-temperature benchmarks including Ising, Potts, and the copper-gold binary alloy, MetaDNS reproduces the thermodynamic distribution. Compared to MCMC-based metadynamics, MetaDNS also achieves comparable exploration requiring fewer bias deposition steps.