Detailed balance in large language model-driven agents

TL;DR

This paper uncovers a detailed balance principle in large language model-driven agents, suggesting they implicitly learn potential functions governing their generative behavior, which could unify understanding across different models.

Contribution

It introduces a novel theoretical framework based on the least action principle to analyze LLM dynamics and discovers a macroscopic physical law in their generative processes.

Findings

Discovery of detailed balance in LLM-generated transitions

Evidence that LLMs learn underlying potential functions

First identification of a macroscopic physical law in LLM dynamics

Abstract

Large language model (LLM)-driven agents are emerging as a powerful new paradigm for solving complex problems. Despite the empirical success of these practices, a theoretical framework to understand and unify their macroscopic dynamics remains lacking. This Letter proposes a method based on the least action principle to estimate the underlying generative directionality of LLMs embedded within agents. By experimentally measuring the transition probabilities between LLM-generated states, we statistically discover a detailed balance in LLM-generated transitions, indicating that LLM generation may not be achieved by generally learning rule sets and strategies, but rather by implicitly learning a class of underlying potential functions that may transcend different LLM architectures and prompt templates. To our knowledge, this is the first discovery of a macroscopic physical law in LLM generative dynamics that does not depend on specific model details. This work is an attempt to establish a macroscopic dynamics theory of complex AI systems, aiming to elevate the study of AI agents from a collection of engineering practices to a science built on effective measurements that are predictable and quantifiable.