Distributed Q-learning-based Shortest-Path Tree Construction in IoT Sensor Networks

TL;DR

This paper introduces a distributed Q-learning approach for constructing shortest-path trees in IoT sensor networks, enabling autonomous, energy-efficient routing that adapts to network changes with high accuracy.

Contribution

The paper presents a novel distributed Q-learning framework for SPT construction in IoT networks, reducing reliance on centralized algorithms and improving scalability and adaptability.

Findings

Achieves over 99% routing accuracy in large networks

Reduces communication overhead compared to traditional methods

Adapts effectively to topology changes

Abstract

Efficient routing in IoT sensor networks is critical for minimizing energy consumption and latency. Traditional centralized algorithms, such as Dijkstra's, are computationally intensive and ill-suited for dynamic, distributed IoT environments. We propose a novel distributed Q-learning framework for constructing shortest-path trees (SPTs), enabling sensor nodes to independently learn optimal next-hop decisions using only local information. States are defined based on node positions and routing history, with a reward function that incentivizes progression toward the sink while penalizing inefficient paths. Trained on diverse network topologies, the framework generalizes effectively to unseen networks. Simulations across 100 to 500 nodes demonstrate near-optimal routing accuracy (over 99% for networks with more than 300 nodes), with minor deviations (1-2 extra hops) in smaller networks having negligible impact on performance. Compared to centralized and flooding-based methods, our approach reduces communication overhead, adapts to topology changes, and enhances scalability and energy efficiency. This work underscores the potential of Q-learning for autonomous, robust routing in resource-constrained IoT networks, offering a scalable alternative to traditional protocols.