Queueing Matching Bandits with Preference Feedback

TL;DR

This paper introduces algorithms for queueing systems with preference-based server-job matching, balancing system stability and learning unknown service rates, with proven bounds and experimental validation.

Contribution

It proposes UCB and Thompson Sampling algorithms for queueing matching with preference feedback, achieving stability and sublinear regret bounds.

Findings

Algorithms stabilize queues with bounded average length.

Regret bounds are sublinear in time horizon.

Experimental results confirm theoretical performance.

Abstract

In this study, we consider multi-class multi-server asymmetric queueing systems consisting of $N$ queues on one side and $K$ servers on the other side, where jobs randomly arrive in queues at each time. The service rate of each job-server assignment is unknown and modeled by a feature-based Multi-nomial Logit (MNL) function. At each time, a scheduler assigns jobs to servers, and each server stochastically serves at most one job based on its preferences over the assigned jobs. The primary goal of the algorithm is to stabilize the queues in the system while learning the service rates of servers. To achieve this goal, we propose algorithms based on UCB and Thompson Sampling, which achieve system stability with an average queue length bound of $O(\min\{N,K\}/\epsilon)$ for a large time horizon $T$, where $\epsilon$ is a traffic slackness of the system. Furthermore, the algorithms achieve sublinear regret bounds of $\tilde{O}(\min\{\sqrt{T} Q_{\max},T^{3/4}\})$, where $Q_{\max}$ represents the maximum queue length over agents and times. Lastly, we provide experimental results to demonstrate the performance of our algorithms.