HRL-TSCH: A Hierarchical Reinforcement Learning-based TSCH Scheduler for IIoT

TL;DR

This paper presents HRL-TSCH, a hierarchical reinforcement learning-based scheduler for IIoT networks that optimizes throughput, power efficiency, and delay, outperforming existing methods.

Contribution

It introduces a novel HRL-based approach for TSCH scheduling in IIoT, with dual policies for link management and resource assignment, addressing multi-objective optimization.

Findings

HRL-TSCH outperforms existing schedulers in simulations.

It effectively balances throughput, power, and delay.

The approach adapts to application-specific requirements.

Abstract

The Industrial Internet of Things (IIoT) demands adaptable Networked Embedded Systems (NES) for optimal performance. Combined with recent advances in Artificial Intelligence (AI), tailored solutions can be developed to meet specific application requirements. This study introduces HRL-TSCH, an approach rooted in Hierarchical Reinforcement Learning (HRL), to devise Time Slotted Channel Hopping (TSCH) schedules provisioning IIoT demand. HRL-TSCH employs dual policies: one at a higher level for TSCH schedule link management, and another at a lower level for timeslot and channel assignments. The proposed RL agents address a multi-objective problem, optimizing throughput, power efficiency, and network delay based on predefined application requirements. Simulation experiments demonstrate HRL-TSCH superiority over existing state-of-art approaches, effectively achieving an optimal balance between throughput, power consumption, and delay, thereby enhancing IIoT network performance.