Coordinated Multi-Agent Imitation Learning

TL;DR

This paper introduces a joint imitation learning approach that infers latent coordination among multiple agents, improving the modeling of complex team behaviors in sports by integrating unsupervised structure learning with policy learning.

Contribution

The paper presents a novel method combining unsupervised structure learning with imitation learning to model coordination as a latent variable in multi-agent systems.

Findings

Coordination inference improves imitation accuracy.

Method effectively models multi-agent team strategies.

Significant performance gains over baseline methods.

Abstract

We study the problem of imitation learning from demonstrations of multiple coordinating agents. One key challenge in this setting is that learning a good model of coordination can be difficult, since coordination is often implicit in the demonstrations and must be inferred as a latent variable. We propose a joint approach that simultaneously learns a latent coordination model along with the individual policies. In particular, our method integrates unsupervised structure learning with conventional imitation learning. We illustrate the power of our approach on a difficult problem of learning multiple policies for fine-grained behavior modeling in team sports, where different players occupy different roles in the coordinated team strategy. We show that having a coordination model to infer the roles of players yields substantially improved imitation loss compared to conventional baselines.