Learning Empirical Evidence Equilibria under Weak Environmental Coupling

TL;DR

This paper introduces the Empirical Evidence Equilibrium (EEE) framework for multi-agent systems with limited observations, analyzing how weak environmental coupling leads to stable equilibria.

Contribution

It formalizes the EEE concept for bounded rationality in decentralized multi-agent environments and proves convergence under weak coupling conditions.

Findings

EEE emerges under weak coupling between agents and environment

Contraction results established for softmax policies

Stable equilibria characterized despite decentralized decision-making

Abstract

Strategic multi-agent systems are fundamentally characterized by decentralization, uncertainty, and ambiguity. Agents operating under limited observations will often need to make decisions based on simplified internal models of the environment, reflecting bounded rationality in both computational capacity and environmental knowledge. The Empirical Evidence Equilibrium (EEE) framework explicitly accounts for these limitations by modeling each agent as forming a potentially misspecified belief derived from signals obtained through partial observations of the environment. The resulting equilibrium concept captures the system's steady state under bounded rationality and decentralization. In this work, we study games in which the environment dynamics are driven jointly by exogenous factors and agents' actions. We analyze agent behavior under Q-value iteration where each agent independently forms a belief model, computes Q-values, and derives a greedy strategy, yet the collective actions of all agents jointly shape the environment each agent faces at the next stage. We prove that despite this decentralization, an EEE emerges from the joint dynamics when the coupling between agents' actions and the environment is sufficiently weak. We further extend this result to softmax policies, establishing a contraction result under a sufficient coupling condition.