Learning-at-Criticality in Large Language Models for Quantum Field Theory and Beyond

TL;DR

This paper introduces Learning at Criticality (LaC), a reinforcement learning scheme that tunes large language models to a phase transition point, enabling peak generalization from minimal data in complex symbolic reasoning tasks like quantum field theory.

Contribution

The paper presents LaC, a novel RL method that operates LLMs at a critical point to enhance learning efficiency and generalization in data-scarce, complex symbolic reasoning tasks.

Findings

LLMs achieve peak generalization at a critical transition point.

LaC enables small models to outperform larger models on quantum field theory problems.

The minimal concept-network model exhibits phase transition characteristics similar to LLMs.

Abstract

Fundamental physics often confronts complex symbolic problems with few guiding exemplars or established principles. While artificial intelligence (AI) offers promise, its typical need for vast datasets to learn from hinders its use in these information-scarce frontiers. We introduce learning at criticality (LaC), a reinforcement learning (RL) scheme that tunes Large Language Models (LLMs) to a sharp learning transition, addressing this information scarcity. At this transition, LLMs achieve peak generalization from minimal data, exemplified by 7-digit base-7 addition -- a test of nontrivial arithmetic reasoning. To elucidate this peak, we analyze a minimal concept-network model (CoNet) designed to capture the essence of how LLMs might link tokens. Trained on a single exemplar, this model also undergoes a sharp learning transition. This transition exhibits hallmarks of a second-order phase transition, notably power-law distributed solution path lengths. At this critical point, the system maximizes a ``critical thinking pattern" crucial for generalization, enabled by the underlying scale-free exploration. This suggests LLMs reach peak performance by operating at criticality, where such explorative dynamics enable the extraction of underlying operational rules. We demonstrate LaC in quantum field theory: an 8B-parameter LLM, tuned to its critical point by LaC using a few exemplars of symbolic Matsubara sums, solves unseen, higher-order problems, significantly outperforming far larger models. LaC thus leverages critical phenomena, a physical principle, to empower AI for complex, data-sparse challenges in fundamental physics.