Learning from All Vehicles

TL;DR

This paper introduces a system that trains driving policies using experiences from all observed vehicles, leveraging their behaviors to create diverse scenarios and improve autonomous driving performance.

Contribution

It proposes a novel approach to learn from all vehicles' behaviors without sensor data, enhancing diversity and reasoning in driving policy training.

Findings

Outperforms prior methods on CARLA leaderboard

Improves driving score by 25 points

Increases route completion rate by 24 points

Abstract

In this paper, we present a system to train driving policies from experiences collected not just from the ego-vehicle, but all vehicles that it observes. This system uses the behaviors of other agents to create more diverse driving scenarios without collecting additional data. The main difficulty in learning from other vehicles is that there is no sensor information. We use a set of supervisory tasks to learn an intermediate representation that is invariant to the viewpoint of the controlling vehicle. This not only provides a richer signal at training time but also allows more complex reasoning during inference. Learning how all vehicles drive helps predict their behavior at test time and can avoid collisions. We evaluate this system in closed-loop driving simulations. Our system outperforms all prior methods on the public CARLA Leaderboard by a wide margin, improving driving score by 25 and route completion rate by 24 points. Our method won the 2021 CARLA Autonomous Driving challenge. Code and data are available at https://github.com/dotchen/LAV.