Implicit Sensing in Traffic Optimization: Advanced Deep Reinforcement Learning Techniques

TL;DR

This paper introduces an integrated deep reinforcement learning system for autonomous vehicles to make lane-changing decisions during highway construction, utilizing MEC-assisted training and simulation for improved traffic management.

Contribution

It presents a novel integrated car-following and lane-changing model using Deep Reinforcement Learning with MEC-assisted training, addressing practical driving scenarios.

Findings

DQN with ε-greedy policy outperforms Boltzmann policy.

MEC-assisted architecture reduces training delay and computational load.

Simulation results validate the effectiveness of the proposed model.

Abstract

A sudden roadblock on highways due to many reasons such as road maintenance, accidents, and car repair is a common situation we encounter almost daily. Autonomous Vehicles (AVs) equipped with sensors that can acquire vehicle dynamics such as speed, acceleration, and location can make intelligent decisions to change lanes before reaching a roadblock. A number of literature studies have examined car-following models and lane-changing models. However, only a few studies proposed an integrated car-following and lane-changing model, which has the potential to model practical driving maneuvers. Hence, in this paper, we present an integrated car-following and lane-changing decision-control system based on Deep Reinforcement Learning (DRL) to address this issue. Specifically, we consider a scenario where sudden construction work will be carried out along a highway. We model the scenario as a Markov Decision Process (MDP) and employ the well-known DQN algorithm to train the RL agent to make the appropriate decision accordingly (i.e., either stay in the same lane or change lanes). To overcome the delay and computational requirement of DRL algorithms, we adopt an MEC-assisted architecture where the RL agents are trained on MEC servers. We utilize the highly reputable SUMO simulator and OPENAI GYM to evaluate the performance of the proposed model under two policies; {\epsilon}-greedy policy and Boltzmann policy. The results unequivocally demonstrate that the DQN agent trained using the {\epsilon}-greedy policy significantly outperforms the one trained with the Boltzmann policy.