Behaviorally Diverse Traffic Simulation via Reinforcement Learning

TL;DR

This paper presents a reinforcement learning-based approach to generate diverse and high-quality traffic behaviors in simulation, improving the realism and variability of autonomous driving scenarios.

Contribution

It introduces a novel, easily-tunable policy generation algorithm that balances behavioral diversity and driving skill using deep reinforcement learning and intrinsic rewards.

Findings

Effective in creating diverse traffic behaviors

Improves behavioral coverage in simulation scenarios

Validated on challenging intersection scenes

Abstract

Traffic simulators are important tools in autonomous driving development. While continuous progress has been made to provide developers more options for modeling various traffic participants, tuning these models to increase their behavioral diversity while maintaining quality is often very challenging. This paper introduces an easily-tunable policy generation algorithm for autonomous driving agents. The proposed algorithm balances diversity and driving skills by leveraging the representation and exploration abilities of deep reinforcement learning via a distinct policy set selector. Moreover, we present an algorithm utilizing intrinsic rewards to widen behavioral differences in the training. To provide quantitative assessments, we develop two trajectory-based evaluation metrics which measure the differences among policies and behavioral coverage. We experimentally show the effectiveness of our methods on several challenging intersection scenes.