Bench4Merge: A Comprehensive Benchmark for Merging in Realistic Dense Traffic with Micro-Interactive Vehicles

TL;DR

Bench4Merge introduces a comprehensive, realistic benchmark for evaluating autonomous vehicle merging in dense traffic, utilizing diverse micro-behavioral vehicle models and LLM-based assessment to improve evaluation accuracy.

Contribution

The paper presents a novel closed-loop benchmarking framework with diverse micro-behavioral vehicle models and LLM-based evaluation, addressing limitations of existing simulators and metrics.

Findings

Benchmark reveals common issues in current merging methods.

Extensive experiments validate the benchmark's effectiveness.

Deployment demonstrates practical applicability.

Abstract

While the capabilities of autonomous driving have advanced rapidly, merging into dense traffic remains a significant challenge, many motion planning methods for this scenario have been proposed but it is hard to evaluate them. Most existing closed-loop simulators rely on rule-based controls for other vehicles, which results in a lack of diversity and randomness, thus failing to accurately assess the motion planning capabilities in highly interactive scenarios. Moreover, traditional evaluation metrics are insufficient for comprehensively evaluating the performance of merging in dense traffic. In response, we proposed a closed-loop evaluation benchmark for assessing motion planning capabilities in merging scenarios. Our approach involves other vehicles trained in large scale datasets with micro-behavioral characteristics that significantly enhance the complexity and diversity. Additionally, we have restructured the evaluation mechanism by leveraging Large Language Models (LLMs) to assess each autonomous vehicle merging onto the main lane. Extensive experiments and test-vehicle deployment have demonstrated the progressiveness of this benchmark. Through this benchmark, we have obtained an evaluation of existing methods and identified common issues. The simulation environment and evaluation process can be accessed at https://github.com/WZM5853/Bench4Merge.