Large Language Model-assisted Autonomous Vehicle Recovery from Immobilization

TL;DR

This paper presents StuckSolver, a novel LLM-based framework that enables autonomous vehicles to recover from immobilization scenarios through self-reasoning and passenger guidance, improving recovery efficiency without modifying existing vehicle architectures.

Contribution

Introduces StuckSolver, a plug-in LLM-driven recovery system for AVs that interfaces with existing stacks to enhance immobilization recovery capabilities.

Findings

Achieves near-state-of-the-art recovery performance autonomously.

Improves recovery success with passenger-guided decision-making.

Operates without modifying existing AV perception and control systems.

Abstract

Despite significant advancements in recent decades, autonomous vehicles (AVs) continue to face challenges in navigating certain traffic scenarios where human drivers excel. In such situations, AVs often become immobilized, disrupting overall traffic flow. Current recovery solutions, such as remote intervention (which is costly and inefficient) and manual takeover (which excludes non-drivers and limits AV accessibility), are inadequate. This paper introduces StuckSolver, a novel Large Language Model (LLM) driven recovery framework that enables AVs to resolve immobilization scenarios through self-reasoning and/or passenger-guided decision-making. StuckSolver is designed as a plug-in add-on module that operates on top of the AV's existing perception-planning-control stack, requiring no modification to its internal architecture. Instead, it interfaces with standard sensor data streams to detect immobilization states, interpret environmental context, and generate high-level recovery commands that can be executed by the AV's native planner. We evaluate StuckSolver on the Bench2Drive benchmark and in custom-designed uncertainty scenarios. Results show that StuckSolver achieves near-state-of-the-art performance through autonomous self-reasoning alone and exhibits further improvements when passenger guidance is incorporated.