Cooperative Resource Trading for Network Slicing in Industrial IoT: A Multi-Agent DRL Approach

TL;DR

This paper proposes a multi-agent deep reinforcement learning approach to facilitate cooperative resource trading in network slicing for industrial IoT, enabling fair and efficient resource sharing among stakeholders.

Contribution

It introduces a novel economic model combining a cooperative Stackelberg game with MADRL for dynamic resource pricing and sharing in IIoT networks.

Findings

The proposed MADRL method converges reliably.

It outperforms baseline methods in utility maximization.

The approach enables fair resource trading among tenants.

Abstract

The industrial Internet of Things (IIoT) and network slicing (NS) paradigms have been envisioned as key enablers for flexible and intelligent manufacturing in the industry 4.0, where a myriad of interconnected machines, sensors, and devices of diversified quality of service (QoS) requirements coexist. To optimize network resource usage, stakeholders in the IIoT network are encouraged to take pragmatic steps towards resource sharing. However, resource sharing is only attractive if the entities involved are able to settle on a fair exchange of resource for remuneration in a win-win situation. In this paper, we design an economic model that analyzes the multilateral strategic trading interactions between sliced tenants in IIoT networks. We formulate the resource pricing and purchasing problem of the seller and buyer tenants as a cooperative Stackelberg game. Particularly, the cooperative game enforces collaboration among the buyer tenants by coalition formation in order to strengthen their position in resource price negotiations as opposed to acting individually, while the Stackelberg game determines the optimal policy optimization of the seller tenants and buyer tenant coalitions. To achieve a Stackelberg equilibrium (SE), a multi-agent deep reinforcement learning (MADRL) method is developed to make flexible pricing and purchasing decisions without prior knowledge of the environment. Simulation results and analysis prove that the proposed method achieves convergence and is superior to other baselines, in terms of utility maximization.