Towards Verifiable and Self-Correcting AI Physicists for Quantum Many-Body Simulations

TL;DR

This paper introduces QMP-Bench, a challenging quantum many-body simulation benchmark, and PhysVEC, a multi-agent framework that ensures verifiable, error-corrected AI research, advancing trustworthy automated scientific discovery.

Contribution

It presents a new benchmark for quantum physics tasks and a multi-agent system that enforces self-verification and error correction in AI research workflows.

Findings

PhysVEC outperforms existing LLMs on QMP-Bench tasks.

PhysVEC achieves reliable physical reproductions from AI generations.

The framework scales favorably at inference time.

Abstract

While large language models (LLMs) promise to revolutionize automated scientific discovery, their application in rigorous real-world physical research is stalled by two critical barriers: a lack of realistic evaluation benchmarks and systemic LLM hallucinations. Here, we address both problems. We introduce QMP-Bench, a pioneering end-to-end research-level benchmark in quantum many-body simulation consisting of $100$ tasks extracted from $21$ high-impact prestigious journals, presenting a challenge even for current frontier LLMs. To establish a paradigm for reliable and transparent AI physicists, we present PhysVEC, a multi-agent framework that enforces self-verifiable and error correction in AI research. PhysVEC seamlessly integrates programming and scientific verifiers to guarantee coding correctness and principle-based physical validity, yielding interpretable evidence and error correction at each step. PhysVEC significantly outperforms existing LLM baselines on various scenarios in QMP-Bench and presents a favorable inference-time scaling, successfully transforming unreliable AI generations into accurate physical reproductions, paving a robust and trustworthy path towards future automated scientific discovery.