IDGraphs: Intrusion Detection and Analysis Using Stream Compositing

TL;DR

IDGraphs is an interactive visualization system that enhances intrusion detection by enabling detailed analysis of flow-level network traffic, revealing complex attack patterns and anomalies in large-scale data.

Contribution

It introduces a novel visualization approach combining flow-level traces and Histographs for effective detection and analysis of network intrusions and anomalies.

Findings

Successfully detected port scanning, worm outbreaks, and TCP floodings.

Analyzed 179 million flow records totaling 1.16TB of data.

Enabled interactive querying and correlation analysis of network attacks.

Abstract

Traffic anomalies and attacks are commonplace in today's networks and identifying them rapidly and accurately is critical for large network operators. For a statistical intrusion detection system (IDS), it is crucial to detect at the flow-level for accurate detection and mitigation. However, existing IDS systems offer only limited support for 1) interactively examining detected intrusions and anomalies, 2) analyzing worm propagation patterns, 3) and discovering correlated attacks. These problems are becoming even more acute as the traffic on today's high-speed routers continues to grow. IDGraphs is an interactive visualization system for intrusion detection that addresses these challenges. The central visualization in the system is a flow-level trace plotted with time on the horizontal axis and aggregated number of unsuccessful connections on the vertical axis. We then summarize a stack of tens or hundreds of thousands of these traces using the Histographs [RW05] technique, which maps data frequency at each pixel to brightness. Users may then interactively query the summary view, performing analysis by highlighting subsets of the traces. For example, brushing a linked correlation matrix view highlights traces with similar patterns, revealing distributed attacks that are difficult to detect using standard statistical analysis. We apply IDGraphs system to a real network router data-set with 179M flow-level records representing a total traffic of 1.16TB. The system successfully detects and analyzes a variety of attacks and anomalies, including port scanning, worm outbreaks, stealthy TCP SYN floodings, and some distributed attacks.