DTC: Real-Time and Accurate Distributed Triangle Counting in Fully Dynamic Graph Streams

TL;DR

This paper introduces DTC, a family of distributed algorithms for real-time, approximate triangle counting in fully dynamic graph streams, effectively handling edge insertions and deletions with high accuracy.

Contribution

The paper presents DTC-AR and DTC-FD algorithms that provide unbiased, accurate, and scalable triangle counting in dynamic graph streams, addressing limitations of prior methods.

Findings

DTC-AR improves accuracy by up to 2029.4 times over baselines.

DTC-FD reduces estimation errors by up to 32.5 times.

Both algorithms scale linearly with graph stream size.

Abstract

Triangle counting is a fundamental problem in graph mining, essential for analyzing graph streams with arbitrary edge orders. However, exact counting becomes impractical due to the massive size of real-world graph streams. To address this, approximate algorithms have been developed, but existing distributed streaming algorithms lack adaptability and struggle with edge deletions. In this article, we propose DTC, a novel family of single-pass distributed streaming algorithms for global and local triangle counting in fully dynamic graph streams. Our DTC-AR algorithm accurately estimates triangle counts without prior knowledge of graph size, leveraging multi-machine resources. Additionally, we introduce DTC-FD, an algorithm tailored for fully dynamic graph streams, incorporating edge insertions and deletions. Using Random Pairing and future edge insertion compensation, DTC-FD achieves unbiased and accurate approximations across multiple machines. Experimental results demonstrate significant improvements over baselines. DTC-AR achieves up to $2029.4\times$ and $27.1\times$ more accuracy, while maintaining the best trade-off between accuracy and storage space. DTC-FD reduces estimation errors by up to $32.5\times$ and $19.3\times$, scaling linearly with graph stream size. These findings highlight the effectiveness of our proposed algorithms in tackling triangle counting in real-world scenarios. The source code and datasets are released and available at \href{https://github.com/wayne4s/srds-dtc.git}{https://github.com/wayne4s/srds-dtc.git}.