Multi-agent Coordination via Flow Matching

TL;DR

MAC-Flow introduces a flow-based framework for multi-agent coordination that balances rich offline behavior representation with real-time efficiency, outperforming diffusion-based methods in speed while maintaining strong performance.

Contribution

The paper proposes MAC-Flow, a novel approach that combines flow-based joint behavior modeling with decentralized policies for fast, effective multi-agent coordination.

Findings

Achieves 14.5x faster inference than diffusion-based MARL methods.

Maintains competitive performance with prior Gaussian policy-based methods.

Validated across 12 environments and 34 datasets.

Abstract

This work presents MAC-Flow, a simple yet expressive framework for multi-agent coordination. We argue that requirements of effective coordination are twofold: (i) a rich representation of the diverse joint behaviors present in offline data and (ii) the ability to act efficiently in real time. However, prior approaches often sacrifice one for the other, i.e., denoising diffusion-based solutions capture complex coordination but are computationally slow, while Gaussian policy-based solutions are fast but brittle in handling multi-agent interaction. MAC-Flow addresses this trade-off by first learning a flow-based representation of joint behaviors, and then distilling it into decentralized one-step policies that preserve coordination while enabling fast execution. Across four different benchmarks, including $12$ environments and $34$ datasets, MAC-Flow alleviates the trade-off between performance and computational cost, specifically achieving about $\boldsymbol{\times14.5}$ faster inference compared to diffusion-based MARL methods, while maintaining good performance. At the same time, its inference speed is similar to that of prior Gaussian policy-based offline multi-agent reinforcement learning (MARL) methods.