Reinforcement Learning-based Admission Control in Delay-sensitive Service Systems

TL;DR

This paper introduces a reinforcement learning-based admission control method for delay-sensitive service systems that guarantees probabilistic delay bounds using only queue length data, without requiring system model knowledge.

Contribution

It presents a novel RL-based admission controller that ensures delay guarantees in service systems without needing network topology or system parameters.

Findings

Provides probabilistic end-to-end delay bounds

Operates without system model information

Effective in delay-sensitive applications

Abstract

Ensuring quality of service (QoS) guarantees in service systems is a challenging task, particularly when the system is composed of more fine-grained services, such as service function chains. An important QoS metric in service systems is the end-to-end delay, which becomes even more important in delay-sensitive applications, where the jobs must be completed within a time deadline. Admission control is one way of providing end-to-end delay guarantee, where the controller accepts a job only if it has a high probability of meeting the deadline. In this paper, we propose a reinforcement learning-based admission controller that guarantees a probabilistic upper-bound on the end-to-end delay of the service system, while minimizes the probability of unnecessary rejections. Our controller only uses the queue length information of the network and requires no knowledge about the network topology or system parameters. Since long-term performance metrics are of great importance in service systems, we take an average-reward reinforcement learning approach, which is well suited to infinite horizon problems. Our evaluations verify that the proposed RL-based admission controller is capable of providing probabilistic bounds on the end-to-end delay of the network, without using system model information.