Multi-Agent Generative Adversarial Imitation Learning

TL;DR

This paper introduces a novel multi-agent imitation learning framework based on inverse reinforcement learning, along with a practical actor-critic algorithm, enabling imitation of complex multi-agent behaviors in high-dimensional environments.

Contribution

It extends imitation learning to multi-agent Markov games, addressing Nash equilibria and non-stationarity, with a new algorithm demonstrating strong empirical results.

Findings

Effective in high-dimensional multi-agent environments

Handles both cooperative and competitive scenarios

Shows good empirical performance

Abstract

Imitation learning algorithms can be used to learn a policy from expert demonstrations without access to a reward signal. However, most existing approaches are not applicable in multi-agent settings due to the existence of multiple (Nash) equilibria and non-stationary environments. We propose a new framework for multi-agent imitation learning for general Markov games, where we build upon a generalized notion of inverse reinforcement learning. We further introduce a practical multi-agent actor-critic algorithm with good empirical performance. Our method can be used to imitate complex behaviors in high-dimensional environments with multiple cooperative or competing agents.