Evaluating Robustness over High Level Driving Instruction for Autonomous Driving

TL;DR

This paper introduces a benchmark to evaluate the robustness of autonomous driving agents in urban environments, focusing on their ability to handle unexpected high-level commands and maintain safe behavior.

Contribution

It proposes a novel evaluation method and benchmark for assessing the robustness of autonomous driving agents against unpredictable high-level instructions.

Findings

Benchmark effectively measures agent robustness in unexpected scenarios

Agents show varying levels of safety and understanding under different commands

The evaluation highlights areas for improving autonomous driving resilience

Abstract

In recent years, we have witnessed increasingly high performance in the field of autonomous end-to-end driving. In particular, more and more research is being done on driving in urban environments, where the car has to follow high level commands to navigate. However, few evaluations are made on the ability of these agents to react in an unexpected situation. Specifically, no evaluations are conducted on the robustness of driving agents in the event of a bad high-level command. We propose here an evaluation method, namely a benchmark that allows to assess the robustness of an agent, and to appreciate its understanding of the environment through its ability to keep a safe behavior, regardless of the instruction.