Scalable Tail Latency Estimation for Data Center Networks

TL;DR

This paper introduces a fast, scalable method for estimating tail latency in large data center networks by decomposing the problem into parallel link simulations, achieving high accuracy without machine learning training delays.

Contribution

The authors develop a novel decomposition technique that enables rapid, accurate tail latency estimation for large networks without relying on machine learning training.

Findings

Estimates run in 1-2 minutes compared to hours for traditional simulators.

Achieves 99th percentile accuracy within 9% for flow completion times.

Applicable to general traffic matrices and topologies.

Abstract

In this paper, we consider how to provide fast estimates of flow-level tail latency performance for very large scale data center networks. Network tail latency is often a crucial metric for cloud application performance that can be affected by a wide variety of factors, including network load, inter-rack traffic skew, traffic burstiness, flow size distributions, oversubscription, and topology asymmetry. Network simulators such as ns-3 and OMNeT++ can provide accurate answers, but are very hard to parallelize, taking hours or days to answer what if questions for a single configuration at even moderate scale. Recent work with MimicNet has shown how to use machine learning to improve simulation performance, but at a cost of including a long training step per configuration, and with assumptions about workload and topology uniformity that typically do not hold in practice. We address this gap by developing a set of techniques to provide fast performance estimates for large scale networks with general traffic matrices and topologies. A key step is to decompose the problem into a large number of parallel independent single-link simulations; we carefully combine these link-level simulations to produce accurate estimates of end-to-end flow level performance distributions for the entire network. Like MimicNet, we exploit symmetry where possible to gain additional speedups, but without relying on machine learning, so there is no training delay. On large-scale networks where ns-3 takes 11 to 27 hours to simulate five seconds of network behavior, our techniques run in one to two minutes with 99th percentile accuracy within 9% for flow completion times.