VL-DPO: Vision-Language-Guided Finetuning for Preference-Aligned Autonomous Driving

TL;DR

VL-DPO introduces a vision-language-guided finetuning method for autonomous driving that aligns motion forecasting with human preferences, improving performance metrics significantly.

Contribution

The paper proposes a novel framework using vision-language models to automatically generate preference data for finetuning autonomous driving models.

Findings

VL-DPO achieves an 11.94% increase in RFS.

VL-DPO reduces ADE by 10.01%.

VLM-based trajectory selection correlates well with human preferences.

Abstract

The rapid growth of autonomous driving datasets has enabled the scaling of powerful motion forecasting models. While large-scale pretraining provides strong performance, the standard imitation objective may not fully capture the complex nuances of human driving preferences. Meanwhile, recent advances in vision-language models (VLMs) have demonstrated impressive reasoning and commonsense understanding. Building on these capabilities, this paper presents VL-DPO, a vision-language-guided framework that aligns ego-vehicle motion forecasting models with human preferences. Our approach leverages a VLM as a zero-shot reasoner to automatically generate preference pairs from a pretrained model's rollouts, which are then used to finetune the model via Direct Preference Optimization (DPO). We finetune our models on the Waymo Open End-to-End Driving Dataset (WOD-E2E) and evaluate performance against held-out human preference annotations using rater feedback score (RFS) and average displacement error (ADE). Our experiments confirm that the VLM's trajectory selection is a high-quality proxy for human preference. Our final model, VL-DPO, yields an 11.94% increase in RFS and a 10.01% reduction in ADE over the pretrained model.