InVDriver: Intra-Instance Aware Vectorized Query-Based Autonomous Driving Transformer

TL;DR

InVDriver introduces a vectorized query-based autonomous driving system that models intra-instance spatial dependencies with masked self-attention, improving accuracy, safety, and trajectory smoothness in end-to-end driving.

Contribution

It is the first to explicitly incorporate intra-instance spatial dependencies via masked self-attention in vectorized autonomous driving frameworks.

Findings

Achieves state-of-the-art results on nuScenes benchmark.

Improves trajectory smoothness and planning accuracy.

Maintains high computational efficiency.

Abstract

End-to-end autonomous driving with its holistic optimization capabilities, has gained increasing traction in academia and industry. Vectorized representations, which preserve instance-level topological information while reducing computational overhead, have emerged as a promising paradigm. While existing vectorized query-based frameworks often overlook the inherent spatial correlations among intra-instance points, resulting in geometrically inconsistent outputs (e.g., fragmented HD map elements or oscillatory trajectories). To address these limitations, we propose InVDriver, a novel vectorized query-based system that systematically models intra-instance spatial dependencies through masked self-attention layers, thereby enhancing planning accuracy and trajectory smoothness. Across all core modules, i.e., perception, prediction, and planning, InVDriver incorporates masked self-attention mechanisms that restrict attention to intra-instance point interactions, enabling coordinated refinement of structural elements while suppressing irrelevant inter-instance noise. Experimental results on the nuScenes benchmark demonstrate that InVDriver achieves state-of-the-art performance, surpassing prior methods in both accuracy and safety, while maintaining high computational efficiency. Our work validates that explicit modeling of intra-instance geometric coherence is critical for advancing vectorized autonomous driving systems, bridging the gap between theoretical advantages of end-to-end frameworks and practical deployment requirements.