Network slicing for vehicular communications: a multi-agent deep reinforcement learning approach

TL;DR

This paper introduces a multi-agent deep reinforcement learning framework for resource allocation in vehicular networks using network slicing and NOMA, addressing the challenges of high mobility and lack of central control.

Contribution

It proposes a novel DRL-based solution for joint spectrum, power, coverage, and packet selection in vehicular networks, handling NP-hard optimization in a distributed manner.

Findings

The DRL approach outperforms centralized benchmarks in efficiency.

The method is robust against network parameter variations.

It enables practical online distributed resource management.

Abstract

This paper studies the multi-agent resource allocation problem in vehicular networks using non-orthogonal multiple access (NOMA) and network slicing. To ensure heterogeneous service requirements for different vehicles, we propose a network slicing architecture. We focus on a non-cellular network scenario where vehicles communicate by the broadcast approach via the direct device-to-device interface. In such a vehicular network, resource allocation among vehicles is very difficult, mainly due to (i) the rapid variation of wireless channels among highly mobile vehicles and (ii) the lack of a central coordination point. Thus, the possibility of acquiring instantaneous channel state information to perform centralized resource allocation is precluded. The resource allocation problem considered is therefore very complex. It includes not only the usual spectrum and power allocation, but also coverage selection (which target vehicles to broadcast to) and packet selection (which network slice to use). This problem must be solved jointly since selected packets can be overlaid using NOMA and therefore spectrum and power must be carefully allocated for better vehicle coverage. To do so, we provide a optimization approach and study the NP-hardness of the problem. Then, we model the problem using multi-agent Markov decision process. Finally, we use a deep reinforcement learning (DRL) approach to solve the problem. The proposed DRL algorithm is practical because it can be implemented in an online and distributed manner. We show that our approach is robust and efficient when faced with different variations of the network parameters and compared to centralized benchmarks.