QCD in Language Models: What do they really know about QCD?

TL;DR

This paper investigates how well large language models understand Quantum Chromodynamics by analyzing their internal representations and probing their knowledge of key QCD concepts, revealing both capabilities and limitations.

Contribution

It introduces a novel methodology for probing LLMs' understanding of complex physics concepts, specifically QCD, and evaluates several open-source models' knowledge of these principles.

Findings

Models encode some QCD concepts but lack full understanding.

Identification of specific patterns in model representations related to QCD.

Highlighting limitations to guide future model improvements.

Abstract

This study presents an analysis of modern open-source large language models (LLMs) -- including Llama, Qwen, and Gemma -- to evaluate their encoded knowledge of Quantum Chromodynamics (QCD). Through reverse engineering of these models' representations, we uncover the naturally idiosyncratic patterns in how foundational QCD concepts are embedded within their parameter spaces. Our methodology combines targeted probing techniques and knowledge extraction protocols to assess the models' understanding of critical QCD principles like color confinement, asymptotic freedom, and the running coupling constant. This work provides a tool for utilizing LLMs as an assistant in physics research, while also highlighting current limitations in their representation of advanced quantum field theory concepts that future model development should address.