Action and Trajectory Planning for Urban Autonomous Driving with Hierarchical Reinforcement Learning

TL;DR

This paper introduces a hierarchical reinforcement learning approach for urban autonomous driving that effectively handles complex multi-task scenarios with dynamic environments, improving decision-making and trajectory planning.

Contribution

The paper proposes atHRL, a hierarchical RL method that models urban driving tasks and plans trajectories using perception data, outperforming existing RL approaches.

Findings

Significant performance improvements over state-of-the-art RL methods.

Effective handling of multiple driving tasks in complex urban scenarios.

Robust trajectory planning in dynamic environments with other vehicles.

Abstract

Reinforcement Learning (RL) has made promising progress in planning and decision-making for Autonomous Vehicles (AVs) in simple driving scenarios. However, existing RL algorithms for AVs fail to learn critical driving skills in complex urban scenarios. First, urban driving scenarios require AVs to handle multiple driving tasks of which conventional RL algorithms are incapable. Second, the presence of other vehicles in urban scenarios results in a dynamically changing environment, which challenges RL algorithms to plan the action and trajectory of the AV. In this work, we propose an action and trajectory planner using Hierarchical Reinforcement Learning (atHRL) method, which models the agent behavior in a hierarchical model by using the perception of the lidar and birdeye view. The proposed atHRL method learns to make decisions about the agent's future trajectory and computes target waypoints under continuous settings based on a hierarchical DDPG algorithm. The waypoints planned by the atHRL model are then sent to a low-level controller to generate the steering and throttle commands required for the vehicle maneuver. We empirically verify the efficacy of atHRL through extensive experiments in complex urban driving scenarios that compose multiple tasks with the presence of other vehicles in the CARLA simulator. The experimental results suggest a significant performance improvement compared to the state-of-the-art RL methods.