CoFlow: Coordinated Few-Step Flow for Offline Multi-Agent Decision Making

TL;DR

CoFlow introduces a novel architecture for offline multi-agent reinforcement learning that achieves high coordination with minimal inference steps by native joint-coupled velocity fields.

Contribution

The paper proposes CoFlow, combining Coordinated Velocity Attention and Adaptive Coordination Gating, enabling single-pass multi-agent generation with superior coordination and efficiency.

Findings

CoFlow matches or surpasses various baseline methods on multiple benchmarks.

Single-pass inference suffices for high-quality coordination across configurations.

CoFlow achieves state-of-the-art coordination with 1-3 denoising steps.

Abstract

Generative models have emerged as a promising paradigm for offline multi-agent reinforcement learning (MARL), but existing approaches require many iterative sampling steps. Recent few-step acceleration methods either distill a joint teacher into independent students or apply averaged velocity fields independently to each agent. Unfortunately, these few-step approaches hurt inter-agent coordination. We show that the efficiency-coordination trade-off is not inherent: single-pass multi-agent generation can preserve coordination when the velocity field is natively joint-coupled. We propose Coordinated few-step Flow (CoFlow), an architecture that combines Coordinated Velocity Attention (CVA) with Adaptive Coordination Gating. A finite-difference consistency surrogate further replaces memory-prohibitive Jacobian-vector product backpropagation through the averaged velocity field with two stop-gradient forward passes. Across 60 configurations spanning MPE, MA-MuJoCo, and SMAC, CoFlow matches or surpasses Gaussian policies, value-based methods, transformer policies, diffusion models, and prior flow baselines on episodic return. Three independent coordination probes confirm that CoFlow's improvements arise from inter-agent coordination rather than per-agent capacity. A denoising-step sweep shows that single-pass inference suffices on every configuration. CoFlow reaches state-of-the-art coordination quality in 1-3 denoising steps under both centralized and decentralized execution. Project Page: https://guowei-zou.github.io/coflow/