Is attention all you need to solve the correlated electron problem?

TL;DR

This paper investigates using a self-attention neural network as a variational ansatz to solve the correlated electron problem in solids, demonstrating accurate results and scalable parameter requirements.

Contribution

It introduces a self-attention based wavefunction ansatz for many-body electron problems, showing its efficiency and scalability in large-scale simulations.

Findings

Accurate solution for the interacting electron problem in a moiré quantum material.

Number of variational parameters scales roughly as N^2 with electrons.

Self-attention ansatz offers an unbiased and efficient approach.

Abstract

The attention mechanism has transformed artificial intelligence research by its ability to learn relations between objects. In this work, we explore how a many-body wavefunction ansatz constructed from a large-parameter self-attention neural network can be used to solve the interacting electron problem in solids. By a systematic neural-network variational Monte Carlo study on a moir\'e quantum material, we demonstrate that the self-attention ansatz provides an accurate and efficient solution without human bias. Moreover, our numerical study finds that the required number of variational parameters scales roughly as $N^2$ with the number of electrons, which opens a path towards efficient large-scale simulations.