Accelerating Structured Chain-of-Thought in Autonomous Vehicles

TL;DR

This paper introduces FastDriveCoT, a parallel decoding method that accelerates structured Chain-of-Thought reasoning in autonomous vehicles, significantly reducing inference latency while maintaining reasoning quality.

Contribution

We propose a novel parallel decoding approach for structured CoT that decomposes reasoning into sub-tasks, enabling concurrent generation and faster inference in autonomous driving models.

Findings

Achieved 3-4× speedup in CoT generation

Reduced end-to-end latency significantly

Maintained downstream task performance

Abstract

Chain-of-Thought (CoT) reasoning enhances the decision-making capabilities of vision-language-action models in autonomous driving, but its autoregressive nature introduces significant inference latency, making it impractical for real-time applications. To address this, we introduce FastDriveCoT, a novel parallel decoding method that accelerates template-structured CoT. Our approach decomposes the reasoning process into a dependency graph of distinct sub-tasks, such as identifying critical objects and summarizing traffic rules, some of which can be generated in parallel. By generating multiple independent reasoning steps concurrently within a single forward pass, we significantly reduce the number of sequential computations. Experiments demonstrate a 3-4$\times$ speedup in CoT generation and a substantial reduction in end-to-end latency across various model architectures, all while preserving the original downstream task improvements brought by incorporating CoT reasoning.