# Generation of ice states through deep reinforcement learning

## TL;DR

This paper introduces a deep reinforcement learning approach to generate ice states in the square ice model, learning physical constraints without prior knowledge and outperforming traditional algorithms in sampling tasks.

## Contribution

The study develops a novel RL framework that autonomously learns to generate topologically constrained states, applicable to various models with similar constraints.

## Key findings

- RL agent learns ice rule and loop-closing without prior info
- Trained policy effectively samples states in MCMC
- Outperforms baseline loop algorithm in benchmarks

## Abstract

We present a deep reinforcement learning framework where a machine agent is trained to search for a policy to generate a ground state for the square ice model by exploring the physical environment. After training, the agent is capable of proposing a sequence of local moves to achieve the goal. Analysis of the trained policy and the state value function indicates that the ice rule and loop-closing condition are learned without prior knowledge. We test the trained policy as a sampler in the Markov chain Monte Carlo and benchmark against the baseline loop algorithm. This framework can be generalized to other models with topological constraints where generation of constraint-preserving states is difficult.