PROVEX: Enhancing SOC Analyst Trust with Explainable Provenance-Based IDS

TL;DR

This paper introduces PROVEX, an explainable AI framework for graph neural network-based intrusion detection systems, enhancing analyst trust by providing transparent, human-interpretable explanations of detection decisions in system provenance data.

Contribution

PROVEX integrates multiple GNN explanation methods into a temporal graph-based IDS, enabling real-time, interpretable alerts that improve trust and understanding for SOC analysts.

Findings

High-fidelity explanations that preserve detection decisions

Concise, interpretable causal subgraphs identified

Average explanation time of 3-5 seconds per event

Abstract

Modern intrusion detection systems (IDS) leverage graph neural networks (GNNs) to detect malicious activity in system provenance data, but their decisions often remain a black box to analysts. This paper presents a comprehensive XAI framework designed to bridge the trust gap in Security Operations Centers (SOCs) by making graph-based detection transparent. We implement this framework on top of KAIROS, a state-of-the-art temporal graph-based IDS, though our design is applicable to any temporal graph-based detector with minimal adaptation. The complete codebase is available at https://github.com/devang1304/provex.git. We augment the detection pipeline with post-hoc explanations that highlight why an alert was triggered, identifying key causal subgraphs and events. We adapt three GNN explanation methods - GraphMask, GNNExplainer, and a variational temporal GNN explainer (VA-TGExplainer) - to the temporal provenance context. These tools output human-interpretable representations of anomalous behavior, including important edges and uncertainty estimates. Our contributions focus on the practical integration of these explainers, addressing challenges in memory management and reproducibility. We demonstrate our framework on the DARPA CADETS Engagement 3 dataset and show that it produces concise window-level explanations for detected attacks. Our evaluation reveals that the explainers preserve the TGNN's decisions with high fidelity, surfacing critical edges such as malicious file interactions and anomalous netflows. The average explanation overhead is 3-5 seconds per event. By providing insight into the model's reasoning, our framework aims to improve analyst trust and triage speed.