AccuracyTrader: Accuracy-aware Approximate Processing for Low Tail Latency and High Result Accuracy in Cloud Online Services

TL;DR

AccuracyTrader is a novel approach that significantly reduces tail latency in cloud online services while maintaining high result accuracy by intelligently using data synopses to produce quick initial results and refine them efficiently.

Contribution

It introduces a new method that creates data synopses for fast initial results and targeted accuracy improvements, outperforming existing techniques in latency and accuracy trade-offs.

Findings

Reduces tail latency by over 40 times with less than 7% accuracy loss.

Achieves over 13 times lower accuracy loss at the same latency compared to existing approximate methods.

Effective in real service workloads, demonstrating practical benefits.

Abstract

Modern latency-critical online services such as search engines often process requests by consulting large input data spanning massive parallel components. Hence the tail latency of these components determines the service latency. To trade off result accuracy for tail latency reduction, existing techniques use the components responding before a specified deadline to produce approximate results. However, they may skip a large proportion of components when load gets heavier, thus incurring large accuracy losses. This paper presents AccuracyTrader that produces approximate results with small accuracy losses while maintaining low tail latency. AccuracyTrader aggregates information of input data on each component to create a small synopsis, thus enabling all components producing initial results quickly using their synopses. AccuracyTrader also uses synopses to identify the parts of input data most related to arbitrary requests' result accuracy, thus first using these parts to improve the produced results in order to minimize accuracy losses. We evaluated AccuracyTrader using workloads in real services. The results show: (i) AccuracyTrader reduces tail latency by over 40 times with accuracy losses of less than 7% compared to existing exact processing techniques; (ii) when using the same latency, AccuracyTrader reduces accuracy losses by over 13 times comparing to existing approximate processing techniques.