Nudge: Stochastically Improving upon FCFS

TL;DR

This paper introduces Nudge, a new scheduling policy that provably outperforms FCFS across all response time percentiles and tail behaviors for light-tailed job size distributions, challenging previous assumptions of FCFS's optimality.

Contribution

The paper presents Nudge, the first policy proven to stochastically improve upon FCFS at every tail point, demonstrating that FCFS is not strongly asymptotically optimal.

Findings

Nudge outperforms FCFS at every tail point for light-tailed distributions.

Nudge provides a multiplicative tail improvement over FCFS.

FCFS is not strongly asymptotically optimal, contrary to previous beliefs.

Abstract

The First-Come First-Served (FCFS) scheduling policy is the most popular scheduling algorithm used in practice. Furthermore, its usage is theoretically validated: for light-tailed job size distributions, FCFS has weakly optimal asymptotic tail of response time. But what if we don't just care about the asymptotic tail? What if we also care about the 99th percentile of response time, or the fraction of jobs that complete in under one second? Is FCFS still best? Outside of the asymptotic regime, only loose bounds on the tail of FCFS are known, and optimality is completely open. In this paper, we introduce a new policy, Nudge, which is the first policy to provably stochastically improve upon FCFS. We prove that Nudge simultaneously improves upon FCFS at every point along the tail, for light-tailed job size distributions. As a result, Nudge outperforms FCFS for every moment and every percentile of response time. Moreover, Nudge provides a multiplicative improvement over FCFS in the asymptotic tail. This resolves a long-standing open problem by showing that, counter to previous conjecture, FCFS is not strongly asymptotically optimal.