Learning-based Near-optimal Motion Planning for Intelligent Vehicles with Uncertain Dynamics

TL;DR

This paper introduces a novel reinforcement learning algorithm combining Gaussian Process regression and sparse kernel methods for near-optimal, safe, and adaptive motion planning in uncertain vehicle dynamics, demonstrated through extensive simulations and real vehicle tests.

Contribution

It proposes a sparse kernel-based RL algorithm with Gaussian Process regression for online adaptation and near-optimal motion planning under uncertain dynamics in intelligent vehicles.

Findings

Outperforms existing motion planning methods in simulation

Produces safer, less conservative driving actions

Demonstrated effective real-world vehicle planning

Abstract

Motion planning has been an important research topic in achieving safe and flexible maneuvers for intelligent vehicles. However, it remains challenging to realize efficient and optimal planning in the presence of uncertain model dynamics. In this paper, a sparse kernel-based reinforcement learning (RL) algorithm with Gaussian Process (GP) Regression (called GP-SKRL) is proposed to achieve online adaption and near-optimal motion planning performance. In this algorithm, we design an efficient sparse GP regression method to learn the uncertain dynamics. Based on the updated model, a sparse kernel-based policy iteration algorithm with an exponential barrier function is designed to learn the near-optimal planning policies with the capability to avoid dynamic obstacles. Thereby, batch-mode GP-SKRL with online adaption capability can estimate the changing system dynamics. The converged RL policies are then deployed on vehicles efficiently under a safety-aware module. As a result, the produced driving actions are safe and less conservative, and the planning performance has been noticeably improved. Extensive simulation results show that GP-SKRL outperforms several advanced motion planning methods in terms of average cumulative cost, trajectory length, and task completion time. In particular, experiments on a Hongqi E-HS3 vehicle demonstrate that superior GP-SKRL provides a practical planning solution.