Multi-Agent Constraint Factorization Reveals Latent Invariant Solution Structure

TL;DR

This paper provides a formal operator-theoretic explanation for why multi-agent systems with large language models often find invariant solutions through constraint factorization, outperforming single-agent approaches.

Contribution

It introduces a novel formal framework modeling multi-agent systems as composed of constraint-enforcing operators, revealing invariant solution structures and their convergence properties.

Findings

Multi-agent systems converge to invariant solution sets defined by intersecting constraints.

Single agents cannot dynamically access these invariant solutions when applying all constraints simultaneously.

The framework extends to soft constraints using proximal operators and applies to text-based dialog systems.

Abstract

Multi-agent systems (MAS) composed of large language models often exhibit improved problem-solving performance despite operating on identical information. In this work, we provide a formal explanation for this phenomenon grounded in operator theory and constrained optimization. We model each agent as enforcing a distinct family of validity constraints on a shared solution state, and show that a MAS implements a factorized composition of constraint-enforcement operators. Under mild conditions, these dynamics converge to invariant solution sets defined by the intersection of agent constraint sets. Such invariant structures are generally not dynamically accessible to a single agent applying all constraints simultaneously, even when expressive capacity and information are identical. We extend this result from exact constraint enforcement to soft constraints via proximal operators, and apply the formalism to contemporary text-based dialog systems.