Counterfactual VLA: Self-Reflective Vision-Language-Action Model with Adaptive Reasoning

TL;DR

This paper introduces CF-VLA, a self-reflective vision-language-action model for autonomous driving that reasons about and revises planned actions using counterfactual analysis, improving safety and accuracy.

Contribution

The work presents a novel self-reflective framework with counterfactual reasoning for autonomous driving, enabling dynamic action correction based on visual context.

Findings

Trajectory accuracy improved by up to 17.6%

Safety metrics increased by 20.5%

Adaptive reasoning activated in challenging scenarios

Abstract

Recent reasoning-augmented Vision-Language-Action (VLA) models have improved the interpretability of end-to-end autonomous driving by generating intermediate reasoning traces. Yet these models primarily describe what they perceive and intend to do, rarely questioning whether their planned actions are safe or appropriate. This work introduces Counterfactual VLA (CF-VLA), a self-reflective VLA framework that enables the model to reason about and revise its planned actions before execution. CF-VLA first generates time-segmented meta-actions that summarize driving intent, and then performs counterfactual reasoning conditioned on both the meta-actions and the visual context. This step simulates potential outcomes, identifies unsafe behaviors, and outputs corrected meta-actions that guide the final trajectory generation. To efficiently obtain such self-reflective capabilities, we propose a rollout-filter-label pipeline that mines high-value scenes from a base (non-counterfactual) VLA's rollouts and labels counterfactual reasoning traces for subsequent training rounds. Experiments on large-scale driving datasets show that CF-VLA improves trajectory accuracy by up to 17.6%, enhances safety metrics by 20.5%, and exhibits adaptive thinking: it only enables counterfactual reasoning in challenging scenarios. By transforming reasoning traces from one-shot descriptions to causal self-correction signals, CF-VLA takes a step toward self-reflective autonomous driving agents that learn to think before they act.