Quantum Many-Body Physics Calculations with Large Language Models

TL;DR

This paper shows that large language models, with carefully designed prompts, can accurately perform complex quantum physics calculations, specifically the Hartree-Fock method, across multiple research papers, demonstrating potential for automated scientific reasoning.

Contribution

The paper introduces a prompt-based approach enabling LLMs to perform multi-step quantum physics calculations, achieving high accuracy and automating parts of the research process.

Findings

GPT-4 correctly derived the Hartree-Fock Hamiltonian in 13 out of 15 cases.

Average score of 87.5 out of 100 on calculation steps across papers.

Effective methods for information extraction and automatic scoring in scientific calculations.

Abstract

Large language models (LLMs) have demonstrated an unprecedented ability to perform complex tasks in multiple domains, including mathematical and scientific reasoning. We demonstrate that with carefully designed prompts, LLMs can accurately carry out key calculations in research papers in theoretical physics. We focus on a broadly used approximation method in quantum physics: the Hartree-Fock method, requiring an analytic multi-step calculation deriving approximate Hamiltonian and corresponding self-consistency equations. To carry out the calculations using LLMs, we design multi-step prompt templates that break down the analytic calculation into standardized steps with placeholders for problem-specific information. We evaluate GPT-4's performance in executing the calculation for 15 research papers from the past decade, demonstrating that, with correction of intermediate steps, it can correctly derive the final Hartree-Fock Hamiltonian in 13 cases and makes minor errors in 2 cases. Aggregating across all research papers, we find an average score of 87.5 (out of 100) on the execution of individual calculation steps. Overall, the requisite skill for doing these calculations is at the graduate level in quantum condensed matter theory. We further use LLMs to mitigate the two primary bottlenecks in this evaluation process: (i) extracting information from papers to fill in templates and (ii) automatic scoring of the calculation steps, demonstrating good results in both cases. The strong performance is the first step for developing algorithms that automatically explore theoretical hypotheses at an unprecedented scale.