On Diffusion Models for Multi-Agent Partial Observability: Shared Attractors, Error Bounds, and Composite Flow

TL;DR

This paper explores how diffusion models can reconstruct global states in multi-agent systems with partial observability, providing theoretical bounds and a composite process with convergence guarantees.

Contribution

It introduces a novel analysis of diffusion models in Dec-POMDPs, including fixed point characterization, error bounds, and a composite diffusion process with convergence guarantees.

Findings

Diffusion models conditioned on local histories represent possible states as stable fixed points.

Shared fixed points in collectively observable settings correspond to the global state.

A surrogate linear model bounds approximation errors, enabling a convergent composite diffusion process.

Abstract

Multiagent systems grapple with partial observability (PO), and the decentralized POMDP (Dec-POMDP) model highlights the fundamental nature of this challenge. Whereas recent approaches to addressing PO have appealed to deep learning models, providing a rigorous understanding of how these models and their approximation errors affect agents' handling of PO and their interactions remain a challenge. In addressing this challenge, we investigate reconstructing global states from local action-observation histories in Dec-POMDPs using diffusion models. We first find that diffusion models conditioned on local history represent possible states as stable fixed points. In collectively observable (CO) Dec-POMDPs, individual diffusion models conditioned on agents' local histories share a unique fixed point corresponding to the global state, while in non-CO settings, shared fixed points yield a distribution of possible states given joint history. We further find that, with deep learning approximation errors, fixed points can deviate from true states and the deviation is negatively correlated to the Jacobian rank. Inspired by this low-rank property, we bound a deviation by constructing a surrogate linear regression model that approximates the local behavior of a diffusion model. With this bound, we propose a \emph{composite diffusion process} iterating over agents with theoretical convergence guarantees to the true state.