TakeAD: Preference-based Post-optimization for End-to-end Autonomous Driving with Expert Takeover Data

TL;DR

TakeAD is a novel framework that improves end-to-end autonomous driving by fine-tuning policies with expert takeover data, effectively reducing disengagements and enhancing closed-loop performance.

Contribution

It introduces a preference-based post-optimization method combining imitation learning and preference alignment to leverage disengagement data for better autonomous driving policies.

Findings

Outperforms pure imitation learning methods on Bench2Drive benchmark.

Effectively reduces driver disengagements during closed-loop driving.

Enhances recovery strategies in disengagement scenarios.

Abstract

Existing end-to-end autonomous driving methods typically rely on imitation learning (IL) but face a key challenge: the misalignment between open-loop training and closed-loop deployment. This misalignment often triggers driver-initiated takeovers and system disengagements during closed-loop execution. How to leverage those expert takeover data from disengagement scenarios and effectively expand the IL policy's capability presents a valuable yet unexplored challenge. In this paper, we propose TakeAD, a novel preference-based post-optimization framework that fine-tunes the pre-trained IL policy with this disengagement data to enhance the closed-loop driving performance. First, we design an efficient expert takeover data collection pipeline inspired by human takeover mechanisms in real-world autonomous driving systems. Then, this post optimization framework integrates iterative Dataset Aggregation (DAgger) for imitation learning with Direct Preference Optimization (DPO) for preference alignment. The DAgger stage equips the policy with fundamental capabilities to handle disengagement states through direct imitation of expert interventions. Subsequently, the DPO stage refines the policy's behavior to better align with expert preferences in disengagement scenarios. Through multiple iterations, the policy progressively learns recovery strategies for disengagement states, thereby mitigating the open-loop gap. Experiments on the closed-loop Bench2Drive benchmark demonstrate our method's effectiveness compared with pure IL methods, with comprehensive ablations confirming the contribution of each component.