# The Supermarket Model with Known and Predicted Service Times

## TL;DR

This paper explores how incorporating predicted service times, obtained via machine learning, into the supermarket model enhances load balancing efficiency, with simulation results showing significant benefits and practical design insights.

## Contribution

It introduces simulation studies of the supermarket model using predicted service times, highlighting benefits and challenges of integrating machine learning predictions into queue management.

## Key findings

- Predicted service times improve load balancing over random queue selection.
- Using predictions for queue choice is often more beneficial than for service ordering.
- Simulation with real-world data confirms practical advantages of the approach.

## Abstract

The supermarket model refers to a system with a large number of queues, where new customers choose d queues at random and join the one with the fewest customers. This model demonstrates the power of even small amounts of choice, as compared to simply joining a queue chosen uniformly at random, for load balancing systems. In this work we perform simulation-based studies to consider variations where service times for a customer are predicted, as might be done in modern settings using machine learning techniques or related mechanisms. Our primary takeaway is that using even seemingly weak predictions of service times can yield significant benefits over blind First In First Out queueing in this context. However, some care must be taken when using predicted service time information to both choose a queue and order elements for service within a queue; while in many cases using the information for both choosing and ordering is beneficial, in many of our simulation settings we find that simply using the number of jobs to choose a queue is better when using predicted service times to order jobs in a queue. In our simulations, we evaluate both synthetic and real-world workloads--in the latter, service times are predicted by machine learning. Our results provide practical guidance for the design of real-world systems; moreover, we leave many natural theoretical open questions for future work, validating their relevance to real-world situations.