Low-Latency Sliding Window Connectivity

TL;DR

This paper introduces a spanning-tree-based method for low-latency sliding window connectivity queries in streaming graphs, drastically reducing update latency and outperforming existing approaches in speed, memory, and efficiency.

Contribution

It presents a novel spanning tree maintenance technique that eliminates replacement edge searches, enabling faster updates and queries in streaming graph connectivity analysis.

Findings

Up to 458x reduction in index update latency

8x improvement in overall throughput

Significantly less memory usage than state-of-the-art methods

Abstract

Connectivity queries, which check whether vertices belong to the same connected component, are fundamental in graph computations. Sliding window connectivity processes these queries over sliding windows, facilitating real-time streaming graph analytics. However, existing methods struggle with low-latency processing due to the significant overhead of continuously updating index structures as edges are inserted and deleted. We introduce a novel approach that leverages spanning trees to efficiently process queries. The novelty of this method lies in its ability to maintain spanning trees efficiently as window updates occur. Notably, our approach completely eliminates the need for replacement edge searches, a traditional bottleneck in managing spanning trees during edge deletions. We also present several optimizations to maximize the potential of spanning-tree-based indexes. Our comprehensive experimental evaluation shows that index update latency in spanning trees can be reduced by up to $458\times$ while maintaining query performance, leading to an $8\times$ improvement in throughput. Our approach also significantly outperforms the state-of-the-art in both query processing and index updates. Additionally, our methods use significantly less memory and demonstrate consistent efficiency across various settings.