LoRD: Adapting Differentiable Driving Policies to Distribution Shifts

TL;DR

This paper introduces LoRD and multi-task fine-tuning techniques to adapt differentiable driving policies to distribution shifts, improving performance and reducing catastrophic forgetting in self-driving vehicle models.

Contribution

It proposes novel adaptation methods for differentiable autonomy stacks in SDVs, evaluated in closed-loop settings, addressing distribution shifts and catastrophic forgetting.

Findings

LoRD and multi-task fine-tuning improve out-of-distribution driving performance.

Methods reduce catastrophic forgetting by up to 23.33%.

Significant gap identified between open-loop and closed-loop evaluations.

Abstract

Distribution shifts between operational domains can severely affect the performance of learned models in self-driving vehicles (SDVs). While this is a well-established problem, prior work has mostly explored naive solutions such as fine-tuning, focusing on the motion prediction task. In this work, we explore novel adaptation strategies for differentiable autonomy stacks consisting of prediction, planning, and control, perform evaluation in closed-loop, and investigate the often-overlooked issue of catastrophic forgetting. Specifically, we introduce two simple yet effective techniques: a low-rank residual decoder (LoRD) and multi-task fine-tuning. Through experiments across three models conducted on two real-world autonomous driving datasets (nuPlan, exiD), we demonstrate the effectiveness of our methods and highlight a significant performance gap between open-loop and closed-loop evaluation in prior approaches. Our approach improves forgetting by up to 23.33% and the closed-loop OOD driving score by 9.93% in comparison to standard fine-tuning.