SWP: Microsecond Network SLOs Without Priorities

TL;DR

The paper introduces swp, a system that dynamically adjusts switch configurations to meet network tail latency SLOs without priority scheduling, saving significant link capacity compared to FIFO.

Contribution

We develop an efficient simulation and optimization framework that enables real-time tail latency control without priority scheduling in cloud networks.

Findings

swp reduces required link capacity by up to 65% compared to FIFO.

swp maintains tail latency SLOs across diverse workloads.

Priority scheduling can worsen overall delay for non-priority traffic.

Abstract

The increasing use of cloud computing for latency-sensitive applications has sparked renewed interest in providing tight bounds on network tail latency. Achieving this in practice at reasonable network utilization has proved elusive, due to a combination of highly bursty application demand, faster link speeds, and heavy-tailed message sizes. While priority scheduling can be used to reduce tail latency for some traffic, this comes at a cost of much worse delay behavior for all other traffic on the network. Most operators choose to run their networks at very low average utilization, despite the added cost, and yet still suffer poor tail behavior. This paper takes a different approach. We build a system, swp, to help operators (and network designers) to understand and control tail latency without relying on priority scheduling. As network workload changes, swp is designed to give real-time advice on the network switch configurations needed to maintain tail latency objectives for each traffic class. The core of swp is an efficient model for simulating the combined effect of traffic characteristics, end-to-end congestion control, and switch scheduling on service-level objectives (SLOs), along with an optimizer that adjusts switch-level scheduling weights assigned to each class. Using simulation across a diverse set of workloads with different SLOs, we show that to meet the same SLOs as swp provides, FIFO would require 65% greater link capacity, and 79% more for scenarios with tight SLOs on bursty traffic classes.