AI-based Dynamic Schedule Calculation in Time Sensitive Networks using GCN-TD3

TL;DR

This paper introduces GCN-TD3, a novel AI-based approach combining graph neural networks and reinforcement learning to dynamically schedule flows in Time Sensitive Networks, improving latency and jitter management in industrial applications.

Contribution

The paper presents a new GCN-TD3 method that adaptively schedules TSN flows using GCNs and TD3, outperforming existing algorithms and including a fallback ILP scheduler for robustness.

Findings

Achieves 90% dynamic flow admission rate within 4000 epochs

Reduces jitter to as low as 2 microseconds

Outperforms DDQN and DDPG in convergence and scheduling efficiency

Abstract

Offline scheduling in Time Sensitive Networking (TSN) utilizing the Time Aware Shaper (TAS) facilitates optimal deterministic latency and jitter-bounds calculation for Time- Triggered (TT) flows. However, the dynamic nature of traffic in industrial settings necessitates a strategy for adaptively scheduling flows without interrupting existing schedules. Our research identifies critical gaps in current dynamic scheduling methods for TSN and introduces the novel GCN-TD3 approach. This novel approach utilizes a Graph Convolutional Network (GCN) for representing the various relations within different components of TSN and employs the Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm to dynamically schedule any incoming flow. Additionally, an Integer Linear Programming (ILP) based offline scheduler is used both to initiate the simulation and serve as a fallback mechanism. This mechanism is triggered to recalculate the entire schedule when the predefined threshold of Gate Control List(GCL) length exceeds. Comparative analyses demonstrate that GCN-TD3 outperforms existing methods like Deep Double Q-Network (DDQN) and Deep Deterministic Policy Gradient (DDPG), exhibiting convergence within 4000 epochs with a 90\% dynamic TT flow admission rate while maintaining deadlines and reducing jitter to as low as 2us. Finally, two modules were developed for the OMNeT++ simulator, facilitating dynamic simulation to evaluate the methodology.