Delay-Aware Scheduling over mmWave/Sub-6 Dual Interfaces: A Reinforcement Learning Approach

TL;DR

This paper proposes a reinforcement learning-based scheduling strategy for dual-interface transmitters over mmWave and sub-6 bands, aiming to minimize delay amid channel intermittency.

Contribution

It introduces a Q-learning approach for delay-aware scheduling and analyzes the impact of channel state information on policy optimality.

Findings

Q-learning effectively reduces delay in dual-interface systems.

Instantaneous CSI benefits delay reduction when channel transition models are unknown.

Optimal policies depend on the availability of channel state information.

Abstract

We consider a transmitter with mmWave/sub6 dual interfaces. Due to the intermittency of mmWave channel, the transmitter must schedule packets wisely across the interfaces to minimize the average delay by observing the system state. We usethe well-known dynamic programming methods and Q-learning to find the optimal scheduling policy and investigate the influenceof observing CSI on the optimal policy under different levels of knowledge of the environment. We find that only when the channel state transition model is not available, the instantaneousCSI can help in reducing system delay