Sampling two-dimensional spin systems with transformers

TL;DR

This paper introduces an efficient transformer-based neural sampler that generates groups of spins simultaneously, enabling larger system sampling and improved effective sample size in 2D spin models.

Contribution

It proposes a novel group-generation transformer approach with an approximated probability model, enhancing sampling efficiency for large 2D spin systems.

Findings

Successfully sampled 180x180 Ising model systems.

Achieved 20 times larger effective sample size than previous neural samplers.

Demonstrated applicability to the 2D Edwards-Anderson model.

Abstract

Autoregressive Neural Networks based on dense or convolutional layers have recently been shown to be a viable strategy for generating classical spin systems. Unlike these methods, sampling with transformers is commonly considered to be computationally inefficient. In this work, we propose a novel approach to transformer-based neural samplers in which we generate not a single spin per step but groups of spins. As an additional improvement, we construct a model of approximated probabilities, further improving the efficiency of the algorithm. Despite our approach being computationally heavier than dense networks or CNN-based approaches, we were able to sample larger systems of up to $180 \times 180$ spins in case of the Ising model. The Effective Sample Size of our sampler is $\sim 20$ times larger than that of the previous state-of-the-art neural sampler when trained for the $128 \times 128$ Ising model at critical temperature. Finally, we also test our algorithm on the 2D Edwards-Anderson model, where we train $64\times 64$ spin systems.