PC3D: Zero-Shot Cooperation Across Variable Rosters via Personalized Context Distillation

TL;DR

PC3D enables decentralized agents to adapt to varying team sizes in cooperative multi-agent reinforcement learning by distilling personalized coordination contexts from local histories.

Contribution

The paper introduces PC3D, a novel method for training decentralized policies that recover and utilize personalized coordination contexts without online retraining.

Findings

PC3D outperforms baselines on three MARL benchmarks.

It achieves higher returns with both seen and unseen team sizes.

Ablation studies confirm the importance of context distillation and adaptive use.

Abstract

Cooperative multi-agent reinforcement learning often assumes a fixed execution team, yet many decentralized systems must operate with varying numbers of active agents during deployment. We study this setting under episodic roster variation: each episode is executed by a set of homogeneous agents, with the team size varying across episodes. Agents act only from local histories, without execution-time communication, privileged coordinators, or online retraining. Therefore, effective cooperation requires each agent to recover relevant context about the active team and adapt its behavior accordingly. To this end, we propose PC3D (Personalized Central Coordination Context Distillation), a method for training decentralized policies to recover and use personalized coordination context from local interaction histories. During training, a set-structured centralized teacher compresses the active team into coordination tokens and personalizes them into agent-specific contexts, which are distilled into decentralized policies. At execution, each agent predicts its own context from local history and adaptively uses it to condition decision-making. Across three cooperative MARL benchmarks, PC3D achieves higher returns than the evaluated baselines with both seen and unseen roster sizes, and ablations attribute these gains to both context distillation and adaptive context use.