Lyapunov-guided Multi-Agent Reinforcement Learning for Delay-Sensitive Wireless Scheduling

TL;DR

This paper introduces a Lyapunov-guided multi-agent reinforcement learning approach for wireless scheduling that effectively reduces delay jitter and violations in delay-sensitive networks.

Contribution

It combines Lyapunov stability theory with hierarchical MARL to optimize resource allocation and delay performance in wireless scheduling.

Findings

Lower delay jitter compared to baseline algorithms

Reduced delay violation rate

Outperforms Round-Robin and MARL with penalties

Abstract

In this paper, a two-stage intelligent scheduler is proposed to minimize the packet-level delay jitter while guaranteeing delay bound. Firstly, Lyapunov technology is employed to transform the delay-violation constraint into a sequential slot-level queue stability problem. Secondly, a hierarchical scheme is proposed to solve the resource allocation between multiple base stations and users, where the multi-agent reinforcement learning (MARL) gives the user priority and the number of scheduled packets, while the underlying scheduler allocates the resource. Our proposed scheme achieves lower delay jitter and delay violation rate than the Round-Robin Earliest Deadline First algorithm and MARL with delay violation penalty.