Multi-Agent Imitation Learning for Driving Simulation

TL;DR

This paper extends Generative Adversarial Imitation Learning to multi-agent driving scenarios using parameter sharing and curriculum learning, resulting in more realistic and stable multi-agent driving simulations.

Contribution

It introduces PS-GAIL, a novel multi-agent imitation learning method that improves the realism and stability of simulated driving interactions.

Findings

PS-GAIL outperforms single-agent GAIL in multi-agent stability.

Policies generated by PS-GAIL better capture human driving behaviors.

The approach enables more realistic long-term multi-agent driving simulations.

Abstract

Simulation is an appealing option for validating the safety of autonomous vehicles. Generative Adversarial Imitation Learning (GAIL) has recently been shown to learn representative human driver models. These human driver models were learned through training in single-agent environments, but they have difficulty in generalizing to multi-agent driving scenarios. We argue these difficulties arise because observations at training and test time are sampled from different distributions. This difference makes such models unsuitable for the simulation of driving scenes, where multiple agents must interact realistically over long time horizons. We extend GAIL to address these shortcomings through a parameter-sharing approach grounded in curriculum learning. Compared with single-agent GAIL policies, policies generated by our PS-GAIL method prove superior at interacting stably in a multi-agent setting and capturing the emergent behavior of human drivers.