Pow-Wow: A Dataset and Study on Collaborative Communication in Pommerman

TL;DR

This paper introduces Pow-Wow, a dataset capturing human communication in a competitive game environment, and demonstrates how structured communication improves AI agent performance.

Contribution

It provides a new dataset for studying goal-directed human communication in multi-agent settings and offers insights into effective communication strategies for autonomous agents.

Findings

Communication improves agent win rates

Identifies effective communication patterns

Provides a dataset for future research

Abstract

In multi-agent learning, agents must coordinate with each other in order to succeed. For humans, this coordination is typically accomplished through the use of language. In this work we perform a controlled study of human language use in a competitive team-based game, and search for useful lessons for structuring communication protocol between autonomous agents. We construct Pow-Wow, a new dataset for studying situated goal-directed human communication. Using the Pommerman game environment, we enlisted teams of humans to play against teams of AI agents, recording their observations, actions, and communications. We analyze the types of communications which result in effective game strategies, annotate them accordingly, and present corpus-level statistical analysis of how trends in communications affect game outcomes. Based on this analysis, we design a communication policy for learning agents, and show that agents which utilize communication achieve higher win-rates against baseline systems than those which do not.