Driver Dojo: A Benchmark for Generalizable Reinforcement Learning for Autonomous Driving

TL;DR

This paper introduces Driver Dojo, a comprehensive benchmark for evaluating the generalization capabilities of reinforcement learning algorithms in autonomous driving across diverse scenarios and variations.

Contribution

It presents a new flexible benchmark with diverse scenario generators and configurations to assess and improve the generalizability of RL policies for autonomous driving.

Findings

Benchmark reveals current RL methods struggle with generalization.

Design choices like action and observation spaces significantly impact policy performance.

Provides a platform for systematic evaluation and development of generalizable RL algorithms.

Abstract

Reinforcement learning (RL) has shown to reach super human-level performance across a wide range of tasks. However, unlike supervised machine learning, learning strategies that generalize well to a wide range of situations remains one of the most challenging problems for real-world RL. Autonomous driving (AD) provides a multi-faceted experimental field, as it is necessary to learn the correct behavior over many variations of road layouts and large distributions of possible traffic situations, including individual driver personalities and hard-to-predict traffic events. In this paper we propose a challenging benchmark for generalizable RL for AD based on a configurable, flexible, and performant code base. Our benchmark uses a catalog of randomized scenario generators, including multiple mechanisms for road layout and traffic variations, different numerical and visual observation types, distinct action spaces, diverse vehicle models, and allows for use under static scenario definitions. In addition to purely algorithmic insights, our application-oriented benchmark also enables a better understanding of the impact of design decisions such as action and observation space on the generalizability of policies. Our benchmark aims to encourage researchers to propose solutions that are able to successfully generalize across scenarios, a task in which current RL methods fail. The code for the benchmark is available at https://github.com/seawee1/driver-dojo.