Q-Learning-Based Time-Critical Data Aggregation Scheduling in IoT

TL;DR

This paper introduces a Q-learning based framework for efficient, low-latency data aggregation scheduling in IoT networks, outperforming traditional heuristics by dynamically learning optimal policies to reduce delays.

Contribution

It presents a novel Q-learning approach that unifies tree construction and scheduling for IoT data aggregation, improving scalability and reducing latency.

Findings

Achieves up to 10.87% lower latency than heuristic algorithms

Demonstrates robustness in static networks with up to 300 nodes

Provides a scalable, dynamic scheduling solution for delay-sensitive IoT applications

Abstract

Time-critical data aggregation in Internet of Things (IoT) networks demands efficient, collision-free scheduling to minimize latency for applications like smart cities and industrial automation. Traditional heuristic methods, with two-phase tree construction and scheduling, often suffer from high computational overhead and suboptimal delays due to their static nature. To address this, we propose a novel Q-learning framework that unifies aggregation tree construction and scheduling, modeling the process as a Markov Decision Process (MDP) with hashed states for scalability. By leveraging a reward function that promotes large, interference-free batch transmissions, our approach dynamically learns optimal scheduling policies. Simulations on static networks with up to 300 nodes demonstrate up to 10.87% lower latency compared to a state-of-the-art heuristic algorithm, highlighting its robustness for delay-sensitive IoT applications. This framework enables timely insights in IoT environments, paving the way for scalable, low-latency data aggregation.