Technical Report for Argoverse2 Scenario Mining Challenges on Iterative Error Correction and Spatially-Aware Prompting

TL;DR

This paper enhances large language model-based scenario mining for autonomous driving by introducing iterative error correction and spatial prompt engineering, significantly improving accuracy and reliability on the Argoverse 2 dataset.

Contribution

It proposes a fault-tolerant iterative code-generation process and specialized spatial prompt engineering to address LLM errors in scenario mining tasks.

Findings

Achieved a HOTA-Temporal score of 52.37 on the test set.

Demonstrated consistent improvements across multiple LLMs.

Enhanced scenario mining accuracy and robustness.

Abstract

Scenario mining from extensive autonomous driving datasets, such as Argoverse 2, is crucial for the development and validation of self-driving systems. The RefAV framework represents a promising approach by employing Large Language Models (LLMs) to translate natural-language queries into executable code for identifying relevant scenarios. However, this method faces challenges, including runtime errors stemming from LLM-generated code and inaccuracies in interpreting parameters for functions that describe complex multi-object spatial relationships. This technical report introduces two key enhancements to address these limitations: (1) a fault-tolerant iterative code-generation mechanism that refines code by re-prompting the LLM with error feedback, and (2) specialized prompt engineering that improves the LLM's comprehension and correct application of spatial-relationship functions. Experiments on the Argoverse 2 validation set with diverse LLMs-Qwen2.5-VL-7B, Gemini 2.5 Flash, and Gemini 2.5 Pro-show consistent gains across multiple metrics; most notably, the proposed system achieves a HOTA-Temporal score of 52.37 on the official test set using Gemini 2.5 Pro. These results underline the efficacy of the proposed techniques for reliable, high-precision scenario mining.