TAAF: A Trace Abstraction and Analysis Framework Synergizing Knowledge Graphs and LLMs

TL;DR

TAAF is a new framework that combines knowledge graphs and large language models to analyze complex software execution traces, making it easier to extract insights and answer questions from massive trace data.

Contribution

It introduces a novel integration of time-indexed knowledge graphs with LLMs for trace analysis, enabling natural language querying and improved insight extraction.

Findings

Up to 31.2% improvement in answer accuracy

Effective in multi-hop and causal reasoning tasks

Benchmark TraceQA-100 facilitates evaluation of trace analysis methods

Abstract

Execution traces are a critical source of information for understanding, debugging, and optimizing complex software systems. However, traces from OS kernels or large-scale applications like Chrome or MySQL are massive and difficult to analyze. Existing tools rely on predefined analyses, and custom insights often require writing domain-specific scripts, which is an error-prone and time-consuming task. This paper introduces TAAF (Trace Abstraction and Analysis Framework), a novel approach that combines time-indexing, knowledge graphs (KGs), and large language models (LLMs) to transform raw trace data into actionable insights. TAAF constructs a time-indexed KG from trace events to capture relationships among entities such as threads, CPUs, and system resources. An LLM then interprets query-specific subgraphs to answer natural-language questions, reducing the need for manual inspection and deep system expertise. To evaluate TAAF, we introduce TraceQA-100, a benchmark of 100 questions grounded in real kernel traces. Experiments across three LLMs and multiple temporal settings show that TAAF improves answer accuracy by up to 31.2%, particularly in multi-hop and causal reasoning tasks. We further analyze where graph-grounded reasoning helps and where limitations remain, offering a foundation for next-generation trace analysis tools.