Rectify, Don't Regret: Avoiding Pitfalls of Differentiable Simulation in Trajectory Prediction

TL;DR

This paper introduces a detached receding horizon rollout method to improve the robustness of trajectory prediction models in autonomous driving, effectively reducing collisions and avoiding pitfalls of differentiable simulation.

Contribution

We propose a novel detached simulation approach that prevents shortcut learning, enabling models to learn genuine recovery behaviors in trajectory prediction.

Findings

Reduces target collisions by up to 33.24% on nuScenes and DeepScenario datasets.

Decreases collisions by up to 27.74% compared to open-loop baselines.

Improves multi-modal prediction diversity and lane alignment.

Abstract

Current open-loop trajectory models struggle in real-world autonomous driving because minor initial deviations often cascade into compounding errors, pushing the agent into out-of-distribution states. While fully differentiable closed-loop simulators attempt to address this, they suffer from shortcut learning: the loss gradients flow backward through induced state inputs, inadvertently leaking future ground truth information directly into the model's own previous predictions. The model exploits these signals to artificially avoid drift, non-causally "regretting" past mistakes rather than learning genuinely reactive recovery. To address this, we introduce a detached receding horizon rollout. By explicitly severing the computation graph between simulation steps, the model learns genuine recovery behaviors from drifted states, forcing it to "rectify" mistakes rather than non-causally optimizing past predictions. Extensive evaluations on the nuScenes and DeepScenario datasets show our approach yields more robust recovery strategies, reducing target collisions by up to 33.24% compared to fully differentiable closed-loop training at high replanning frequencies. Furthermore, compared to standard open-loop baselines, our non-differentiable framework decreases collisions by up to 27.74% in dense environments while simultaneously improving multi-modal prediction diversity and lane alignment.