Multi-Agent Deep Reinforcement Learning enabled Computation Resource Allocation in a Vehicular Cloud Network

TL;DR

This paper introduces a multi-agent deep reinforcement learning approach for distributed computational resource allocation in vehicular cloud networks, addressing the challenges of dynamic environments without centralized control.

Contribution

It proposes a novel MADDPG-based scheme for resource sharing in vehicular clouds, utilizing a centralized training and decentralized execution framework.

Findings

Effective resource allocation demonstrated in simulations

Improved performance over traditional methods

Robustness in dynamic, decentralized environments

Abstract

In this paper, we investigate the computational resource allocation problem in a distributed Ad-Hoc vehicular network with no centralized infrastructure support. To support the ever increasing computational needs in such a vehicular network, the distributed virtual cloud network (VCN) is formed, based on which a computational resource sharing scheme through offloading among nearby vehicles is proposed. In view of the time-varying computational resource in VCN, the statistical distribution characteristics for computational resource are analyzed in detail. Thereby, a resource-aware combinatorial optimization objective mechanism is proposed. To alleviate the non-stationary environment caused by the typically multi-agent environment in VCN, we adopt a centralized training and decentralized execution framework. In addition, for the objective optimization problem, we model it as a Markov game and propose a DRL based multi-agent deep deterministic reinforcement learning (MADDPG) algorithm to solve it. Interestingly, to overcome the dilemma of lacking a real central control unit in VCN, the allocation is actually completed on the vehicles in a distributed manner. The simulation results are presented to demonstrate our scheme's effectiveness.