Crane: An Accurate and Scalable Neural Sketch for Graph Stream Summarization

TL;DR

Crane introduces a hierarchical neural sketch architecture for graph stream summarization that improves accuracy and scalability by effectively managing frequent and rare items within limited memory constraints.

Contribution

This paper presents Crane, a novel neural sketch with a hierarchical carry mechanism and adaptive memory expansion, enhancing graph stream summarization accuracy and scalability.

Findings

Reduces estimation error by approximately 10x compared to existing methods.

Effectively manages frequent and infrequent items within limited memory.

Demonstrates scalability across datasets from 20K to 60M edges.

Abstract

Graph streams are rapidly evolving sequences of edges that convey continuously changing relationships among entities, playing a crucial role in domains such as networking, finance, and cybersecurity. Their massive scale and high dynamism make obtaining accurate statistics challenging with limited memory constraints. Traditional methods summarize graph streams through hand-crafted sketches, while recent studies have begun to replace these sketches with neural counterparts to improve adaptability and accuracy. However, this shift faces a major challenge: under limited memory, dominant frequent items tend to overshadow rare ones, hindering the neural network's ability to recover accurate statistics. To address this, we propose Crane, a hierarchical neural sketch architecture for graph stream summarization. Crane uses a hierarchical carry mechanism that automatically elevates frequent items to higher memory layers, reducing interference between frequent and infrequent items within the same layer. To better accommodate real-world deployment, Crane further adopts an adaptive memory expansion strategy that dynamically adds new layers once the occupancy of the top layer exceeds a threshold, enabling scalability across diverse data magnitudes. Extensive experiments on various datasets ranging from 20K to 60M edges demonstrate that Crane reduces estimation error by roughly 10x compared to state-of-the-art methods.