Urban Driving with Conditional Imitation Learning

TL;DR

This paper presents an end-to-end conditional imitation learning approach for urban autonomous driving, integrating lateral and longitudinal control, trained on real human demonstrations, and successfully tested on real European urban streets.

Contribution

It introduces a novel IL method combining lateral and longitudinal control for real urban driving, trained on extensive real-world data, and evaluated in real-world conditions.

Findings

Successfully drove 35km on urban streets using the learned policy.

Addressed dataset bias through data balancing techniques.

Demonstrated real-world applicability beyond simulation.

Abstract

Hand-crafting generalised decision-making rules for real-world urban autonomous driving is hard. Alternatively, learning behaviour from easy-to-collect human driving demonstrations is appealing. Prior work has studied imitation learning (IL) for autonomous driving with a number of limitations. Examples include only performing lane-following rather than following a user-defined route, only using a single camera view or heavily cropped frames lacking state observability, only lateral (steering) control, but not longitudinal (speed) control and a lack of interaction with traffic. Importantly, the majority of such systems have been primarily evaluated in simulation - a simple domain, which lacks real-world complexities. Motivated by these challenges, we focus on learning representations of semantics, geometry and motion with computer vision for IL from human driving demonstrations. As our main contribution, we present an end-to-end conditional imitation learning approach, combining both lateral and longitudinal control on a real vehicle for following urban routes with simple traffic. We address inherent dataset bias by data balancing, training our final policy on approximately 30 hours of demonstrations gathered over six months. We evaluate our method on an autonomous vehicle by driving 35km of novel routes in European urban streets.