Coverage-aware and Reinforcement Learning Using Multi-agent Approach for HD Map QoS in a Realistic Environment

TL;DR

This paper presents a multi-agent reinforcement learning approach using Q-Learning at the application layer to optimize HD map data transmission in VANETs, improving network performance without altering existing standards.

Contribution

It introduces a multi-agent Q-Learning method for HD map QoS in VANETs that operates at the application layer, avoiding standard modifications and outperforming DQN and Actor-Critic methods.

Findings

Multi-agent setup yields higher performance than single-agent.

Q-Learning at application layer improves network efficiency.

Proposed method outperforms DQN and Actor-Critic algorithms.

Abstract

One effective way to optimize the offloading process is by minimizing the transmission time. This is particularly true in a Vehicular Adhoc Network (VANET) where vehicles frequently download and upload High-definition (HD) map data which requires constant updates. This implies that latency and throughput requirements must be guaranteed by the wireless system. To achieve this, adjustable contention windows (CW) allocation strategies in the standard IEEE802.11p have been explored by numerous researchers. Nevertheless, their implementations demand alterations to the existing standard which is not always desirable. To address this issue, we proposed a Q-Learning algorithm that operates at the application layer. Moreover, it could be deployed in any wireless network thereby mitigating the compatibility issues. The solution has demonstrated a better network performance with relatively fewer optimization requirements as compared to the Deep Q Network (DQN) and Actor-Critic algorithms. The same is observed while evaluating the model in a multi-agent setup showing higher performance compared to the single-agent setup.