Trajectory Planning for Autonomous Vehicle Using Iterative Reward Prediction in Reinforcement Learning

TL;DR

This paper introduces an iterative reward prediction reinforcement learning approach for autonomous vehicle trajectory planning, addressing stability and uncertainty issues, and demonstrating significant improvements in simulation.

Contribution

It proposes a novel iterative reward prediction method with uncertainty propagation to improve stability and robustness in reinforcement learning-based trajectory planning.

Findings

Reduced collision rate by 60.17%

Increased average reward by 30.82 times

Validated using CARLA simulator

Abstract

Traditional trajectory planning methods for autonomous vehicles have several limitations. For example, heuristic and explicit simple rules limit generalizability and hinder complex motions. These limitations can be addressed using reinforcement learning-based trajectory planning. However, reinforcement learning suffers from unstable learning, and existing reinforcement learning-based trajectory planning methods do not consider the uncertainties. Thus, this paper, proposes a reinforcement learning-based trajectory planning method for autonomous vehicles. The proposed method involves an iterative reward prediction approach that iteratively predicts expectations of future states. These predicted states are then used to forecast rewards and integrated into the learning process to enhance stability. Additionally, a method is proposed that utilizes uncertainty propagation to make the reinforcement learning agent aware of uncertainties. The proposed method was evaluated using the CARLA simulator. Compared to the baseline methods, the proposed method reduced the collision rate by 60.17 %, and increased the average reward by 30.82 times. A video of the proposed method is available at https://www.youtube.com/watch?v=PfDbaeLfcN4.