Measuring Round-Trip Response Latencies Under Asymmetric Routing

TL;DR

This paper introduces PIRATE, a passive method for estimating client response latencies using only client-to-server traffic, even with encrypted transport headers, enabling better latency monitoring and optimization.

Contribution

PIRATE is a novel passive approach that estimates client response latencies from encrypted traffic, improving latency measurement accuracy without client instrumentation.

Findings

PIRATE estimates latencies within 1% accuracy.

Using PIRATE with a load balancer reduces tail latencies by 37%.

Effective latency measurement with encrypted traffic.

Abstract

Latency is a key indicator of Internet service performance. Continuously tracking the latency of client requests enables service operators to quickly identify bottlenecks, perform adaptive resource allocation or routing, and mitigate attacks. Passively measuring the response latency at intermediate vantage points is attractive since it provides insight into the experience of real clients without requiring client instrumentation or incurring probing overheads. This paper presents PIRATE, a passive approach to measure response latencies when only the client-to-server traffic is visible, even when transport headers are encrypted. PIRATE estimates the time gap between causal pairs - two requests such that the response to the first triggered the second - as a proxy for the client-side response latency. Our experiments with a realistic web application show that PIRATE can estimate the response latencies measured at the client application layer to within 1 percent. A PIRATE-enhanced layer-4 load balancer (with DSR) cuts tail latencies by 37 percent.