Learning to Drive in New Cities Without Human Demonstrations

TL;DR

This paper presents NOMAD, a self-play reinforcement learning method that adapts autonomous driving policies to new cities using only map data, eliminating the need for human demonstrations and reducing data collection costs.

Contribution

Introduces NOMAD, a novel self-play reinforcement learning approach for city transfer in autonomous driving that requires no human demonstrations from the target city.

Findings

Significantly improves task success rate in new cities

Enhances trajectory realism in target environments

Operates effectively with minimal data and simple rewards

Abstract

While autonomous vehicles have achieved reliable performance within specific operating regions, their deployment to new cities remains costly and slow. A key bottleneck is the need to collect many human demonstration trajectories when adapting driving policies to new cities that differ from those seen in training in terms of road geometry, traffic rules, and interaction patterns. In this paper, we show that self-play multi-agent reinforcement learning can adapt a driving policy to a substantially different target city using only the map and meta-information, without requiring any human demonstrations from that city. We introduce NO data Map-based self-play for Autonomous Driving (NOMAD), which enables policy adaptation in a simulator constructed based on the target-city map. Using a simple reward function, NOMAD substantially improves both task success rate and trajectory realism in target cities, demonstrating an effective and scalable alternative to data-intensive city-transfer methods. Project Page: https://nomaddrive.github.io/