Multiclass Queue Scheduling Under Slowdown: An Approximate Dynamic Programming Approach

TL;DR

This paper develops a simulation-based Approximate Dynamic Programming algorithm to optimize scheduling in multiclass queues with wait-dependent slowdowns, improving performance over benchmarks and providing practical insights.

Contribution

It introduces a novel ADP approach that models policies with classifiers, estimates value function differences via coupling, and employs adaptive sampling for efficient optimization.

Findings

ADP policies outperform benchmarks in simulations.

Optimal policies balance immediate cost reduction and system stability.

Case study demonstrates practical effectiveness in healthcare scheduling.

Abstract

In many service systems, especially those in healthcare, customer waiting times can result in increased service requirements. Such service slowdowns can significantly impact system performance. Therefore, it is important to properly account for their impact when designing scheduling policies. Scheduling under wait-dependent service times is challenging, especially when multiple customer classes are heterogeneously affected by waiting. In this work, we study scheduling policies in multiclass, multiserver queues with wait-dependent service slowdowns. We propose a simulation-based Approximate Dynamic Programming (ADP) algorithm to find close-to-optimal scheduling policies. The ADP algorithm (i) represents the policy using classifiers based on the index policy structure, (ii) leverages a coupling method to estimate the differences of the relative value functions directly, and (iii) uses adaptive sampling for efficient state-space exploration. Through extensive numerical experiments, we illustrate that the ADP algorithm generates close-to-optimal policies that outperform well-known benchmarks. We also provide insights into the structure of the optimal policy, which reveals an important trade-off between instantaneous cost reduction and preventing the system from reaching high-cost equilibria. Lastly, we conduct a case study on scheduling admissions into rehabilitation care to illustrate the effectiveness of the ADP algorithm in practice.