Optimal Admission Control for Multiclass Queues with Time-Varying Arrival Rates via State Abstraction

TL;DR

This paper introduces an optimal control framework for multiclass queue admission with time-varying arrivals, leveraging state abstraction to handle complexity and validating the approach on synthetic and real financial data.

Contribution

It formulates a hybrid MDP for the admission control problem, derives its structure, and proposes an approximation method to improve scalability for multiple classes.

Findings

Exact solution to the hybrid MDP is obtained.

Discrete time solution converges to continuous time optimal.

State abstraction improves scalability for many classes.

Abstract

We consider a novel queuing problem where the decision-maker must choose to accept or reject randomly arriving tasks into a no buffer queue which are processed by $N$ identical servers. Each task has a price, which is a positive real number, and a class. Each class of task has a different price distribution and service rate, and arrives according to an inhomogenous Poisson process. The objective is to decide which tasks to accept so that the total price of tasks processed is maximised over a finite horizon. We formulate the problem as a discrete time Markov Decision Process (MDP) with a hybrid state space. We show that the optimal value function has a specific structure, which enables us to solve the hybrid MDP exactly. Moreover, we prove that as the time step is reduced, the discrete time solution approaches the optimal solution to the original continuous time problem. To improve the scalability of our approach to a greater number of task classes, we present an approximation based on state abstraction. We validate our approach on synthetic data, as well as a real financial fraud data set, which is the motivating application for this work.