Can Vehicle Motion Planning Generalize to Realistic Long-tail Scenarios?

TL;DR

This paper introduces a new benchmark for testing autonomous vehicle planners in rare, challenging scenarios and demonstrates that hybrid models combining large language models with rule-based systems outperform existing methods in these complex situations.

Contribution

The paper presents interPlan, a novel closed-loop benchmark for rare driving scenarios, and proposes a hybrid LLM-rule-based planner that achieves superior performance.

Findings

Existing planners struggle with rare scenarios in interPlan.

Hybrid LLM-rule-based planner outperforms pure rule-based and learning-based planners.

interPlan effectively evaluates generalization in autonomous driving.

Abstract

Real-world autonomous driving systems must make safe decisions in the face of rare and diverse traffic scenarios. Current state-of-the-art planners are mostly evaluated on real-world datasets like nuScenes (open-loop) or nuPlan (closed-loop). In particular, nuPlan seems to be an expressive evaluation method since it is based on real-world data and closed-loop, yet it mostly covers basic driving scenarios. This makes it difficult to judge a planner's capabilities to generalize to rarely-seen situations. Therefore, we propose a novel closed-loop benchmark interPlan containing several edge cases and challenging driving scenarios. We assess existing state-of-the-art planners on our benchmark and show that neither rule-based nor learning-based planners can safely navigate the interPlan scenarios. A recently evolving direction is the usage of foundation models like large language models (LLM) to handle generalization. We evaluate an LLM-only planner and introduce a novel hybrid planner that combines an LLM-based behavior planner with a rule-based motion planner that achieves state-of-the-art performance on our benchmark.