Market-Based Model in CR-WSN: A Q-Probabilistic Multi-agent Learning Approach

TL;DR

This paper introduces a novel Q-probabilistic multi-agent reinforcement learning approach for resource allocation in cognitive radio wireless sensor networks, enhancing autonomy and efficiency in smart city environments.

Contribution

It proposes a new Q-probabilistic multi-agent learning algorithm tailored for market-based resource allocation in CR-WSN, modeling primary and secondary users as sellers and buyers.

Findings

The QPML approach effectively allocates resources in CR-WSN.

The method converges rapidly in multi-agent interactions.

Experimental results demonstrate improved spectrum utilization.

Abstract

The ever-increasingly urban populations and their material demands have brought unprecedented burdens to cities. Smart cities leverage emerging technologies like the Internet of Things (IoT), Cognitive Radio Wireless Sensor Network (CR-WSN) to provide better QoE and QoS for all citizens. However, resource scarcity is an important challenge in CR-WSN. Generally, this problem is handled by auction theory or game theory. To make CR-WSN nodes intelligent and more autonomous in resource allocation, we propose a multi-agent reinforcement learning (MARL) algorithm to learn the optimal resource allocation strategy in the oligopoly market model. Firstly, we model a multi-agent scenario, in which the primary users (PUs) is the sellers and the secondary users (SUs) is the buyers. Then, we propose the Q-probabilistic multiagent learning (QPML) and apply it to allocate resources in the market. In the multi-agent interactive learning process, the PUs and SUs learn strategies to maximize their benefits and improve spectrum utilization. Experimental results show the efficiency of our QPML approach, which can also converge quickly.