ConventionPlay: Capability-Limited Training for Robust Ad-Hoc Collaboration

TL;DR

ConventionPlay is a reinforcement learning method that trains agents to adaptively coordinate with diverse partners by probing their capabilities, leading to improved joint strategy effectiveness.

Contribution

It introduces a novel training approach that incorporates capability limits into reinforcement learning for better ad-hoc team coordination.

Findings

Achieves superior coordination efficiency in canonical tasks.

Effective in environments with differentiated convention payoffs.

Learns to probe and adapt to partner capabilities.

Abstract

Ad-hoc collaboration often relies on identifying and adhering to shared conventions. However, when partners can follow multiple conventions, agents must do more than simply adapt; they must actively steer the team toward the most effective joint strategy. We present ConventionPlay, a reinforcement learning-based approach that extends cognitive hierarchies to include a diverse population of adaptive followers. By training against partners with varied capability limits, our agent learns to probe its partner's repertoire, leading the team when possible and following when necessary. Our results in canonical coordination tasks show that ConventionPlay achieves superior coordination efficiency, particularly in settings where conventions have differentiated payoffs.