Reinforcement Learning for Protocol Synthesis in Resource-Constrained Wireless Sensor and IoT Networks

TL;DR

This paper presents a reinforcement learning-based framework for protocol synthesis in resource-constrained wireless sensor and IoT networks, enabling adaptive, collision-avoiding medium access without complex hardware operations.

Contribution

It introduces a novel RL and Multi Armed Bandit approach for MAC protocol synthesis that operates without carrier sensing or synchronization, suitable for ultra simple transceivers.

Findings

Nodes can learn to avoid collisions effectively.

The system achieves throughput comparable to ALOHA protocols.

It sustains high network throughput under heavy traffic loads.

Abstract

This article explores the concepts of online protocol synthesis using Reinforcement Learning (RL). The study is performed in the context of sensor and IoT networks with ultra low complexity wireless transceivers. The paper introduces the use of RL and Multi Armed Bandit (MAB), a specific type of RL, for Medium Access Control (MAC) under different network and traffic conditions. It then introduces a novel learning based protocol synthesis framework that addresses specific difficulties and limitations in medium access for both random access and time slotted networks. The mechanism does not rely on carrier sensing, network time-synchronization, collision detection, and other low level complex operations, thus making it ideal for ultra simple transceiver hardware used in resource constrained sensor and IoT networks. Additionally, the ability of independent protocol learning by the nodes makes the system robust and adaptive to the changes in network and traffic conditions. It is shown that the nodes can be trained to learn to avoid collisions, and to achieve network throughputs that are comparable to ALOHA based access protocols in sensor and IoT networks with simplest transceiver hardware. It is also shown that using RL, it is feasible to synthesize access protocols that can sustain network throughput at high traffic loads, which is not feasible in the ALOHA-based systems. The ability of the system to provide throughput fairness under network and traffic heterogeneities are also experimentally demonstrated.