Iterative Imitation Policy Improvement for Interactive Autonomous Driving

TL;DR

This paper introduces an iterative imitation learning approach for autonomous urban driving, utilizing a weak simulator and a pseudo-expert to enhance policy performance without risky real-world data collection.

Contribution

It presents a novel imitation learning framework combining data aggregation with a weak simulator and pseudo-expert demonstrations for safer, efficient policy improvement in urban driving.

Findings

Significant performance improvements over baseline BC policy.

Effective use of a weak simulator for data collection.

Successful validation in real urban traffic scenarios.

Abstract

We propose an imitation learning system for autonomous driving in urban traffic with interactions. We train a Behavioral Cloning~(BC) policy to imitate driving behavior collected from the real urban traffic, and apply the data aggregation algorithm to improve its performance iteratively. Applying data aggregation in this setting comes with two challenges. The first challenge is that it is expensive and dangerous to collect online rollout data in the real urban traffic. Creating similar traffic scenarios in simulator like CARLA for online rollout collection can also be difficult. Instead, we propose to create a weak simulator from the training dataset, in which all the surrounding vehicles follow the data trajectory provided by the dataset. We find that the collected online data in such a simulator can still be used to improve BC policy's performance. The second challenge is the tedious and time-consuming process of human labelling process during online rollout. To solve this problem, we use an A$^*$ planner as a pseudo-expert to provide expert-like demonstration. We validate our proposed imitation learning system in the real urban traffic scenarios. The experimental results show that our system can significantly improve the performance of baseline BC policy.