Modified DDPG car-following model with a real-world human driving experience with CARLA simulator

TL;DR

This paper introduces a two-stage deep reinforcement learning approach for car-following that incorporates real-world human driving experience, improving autonomous driving performance and human-robot interaction capabilities in simulated environments.

Contribution

A novel two-stage DRL method that integrates real-world human driving data into training, enhancing generalization and practical applicability in autonomous vehicle control.

Findings

The proposed method outperforms pure DRL agents in various scenarios.

Incorporating human driving experience improves efficiency and reasonableness of the autonomous agent.

The agent is more suitable for human-robot interaction in traffic.

Abstract

In the autonomous driving field, fusion of human knowledge into Deep Reinforcement Learning (DRL) is often based on the human demonstration recorded in a simulated environment. This limits the generalization and the feasibility of application in real-world traffic. We propose a two-stage DRL method to train a car-following agent, that modifies the policy by leveraging the real-world human driving experience and achieves performance superior to the pure DRL agent. Training a DRL agent is done within CARLA framework with Robot Operating System (ROS). For evaluation, we designed different driving scenarios to compare the proposed two-stage DRL car-following agent with other agents. After extracting the "good" behavior from the human driver, the agent becomes more efficient and reasonable, which makes this autonomous agent more suitable for Human-Robot Interaction (HRI) traffic.