Autonomous Vehicle Path Planning by Searching With Differentiable Simulation

TL;DR

This paper introduces Differentiable Simulation for Search (DSS), a novel framework that uses a differentiable simulator to improve autonomous vehicle path planning by enabling gradient-based optimization of action sequences.

Contribution

The paper presents DSS, a framework that leverages a differentiable simulator with hardcoded dynamics for accurate and efficient path planning in autonomous driving scenarios.

Findings

DSS significantly improves path planning accuracy.

DSS outperforms sequence prediction and imitation learning.

DSS enhances tracking in complex traffic scenarios.

Abstract

Planning allows an agent to safely refine its actions before executing them in the real world. In autonomous driving, this is crucial to avoid collisions and navigate in complex, dense traffic scenarios. One way to plan is to search for the best action sequence. However, this is challenging when all necessary components - policy, next-state predictor, and critic - have to be learned. Here we propose Differentiable Simulation for Search (DSS), a framework that leverages the differentiable simulator Waymax as both a next state predictor and a critic. It relies on the simulator's hardcoded dynamics, making state predictions highly accurate, while utilizing the simulator's differentiability to effectively search across action sequences. Our DSS agent optimizes its actions using gradient descent over imagined future trajectories. We show experimentally that DSS - the combination of planning gradients and stochastic search - significantly improves tracking and path planning accuracy compared to sequence prediction, imitation learning, model-free RL, and other planning methods.