Visualizing Distributed Traces in Aggregate

TL;DR

This paper introduces a method to aggregate and visualize similar distributed traces in large datasets, aiding developers in system debugging and understanding complex interactions more efficiently.

Contribution

The paper proposes a novel aggregation and visualization technique for distributed traces, enabling better detection of patterns and similarities in large trace datasets.

Findings

Effective grouping of traces based on shared services and structural similarity

Development of an aggregate trace data structure for visualization

Filtering incomplete traces improves analysis accuracy

Abstract

Distributed systems are comprised of many components that communicate together to form an application. Distributed tracing gives us visibility into these complex interactions, but it can be difficult to reason about the system's behavior, even with traces. Systems collect large amounts of tracing data even with low sampling rates. Even when there are patterns in the system, it is often difficult to detect similarities in traces since current tools mainly allow developers to visualize individual traces. Debugging and system optimization is difficult for developers without an understanding of the whole trace dataset. In order to help present these similarities, this paper proposes a method to aggregate traces in a way that groups together and visualizes similar traces. We do so by assigning a few traces that are representative of each set. We suggest that traces can be grouped based on how many services they share, how many levels the graph has, how structurally similar they are, or how close their latencies are. We also develop an aggregate trace data structure as a way to comprehensively visualize these groups and a method for filtering out incomplete traces if a more complete version of the trace exists. The unique traces of each group are especially useful to developers for troubleshooting. Overall, our approach allows for a more efficient method of analyzing system behavior.