CuckooGraph: A Scalable and Space-Time Efficient Data Structure for Large-Scale Dynamic Graphs
Zhuochen Fan, Yalun Cai, Zirui Liu, Jiarui Guo, Xin Fan, Tong Yang,, Bin Cui

TL;DR
CuckooGraph is a new data structure designed for large-scale dynamic graphs that offers high performance and space efficiency by adaptively resizing and handling updates with minimal memory access.
Contribution
It introduces CuckooGraph, a novel adaptive data structure with techniques TRANSFORMATION and DENYLIST, improving scalability and efficiency for dynamic graph processing.
Findings
33x higher insertion throughput than Spruce
Requires only 68% of the memory space of Spruce
Effectively handles dynamic updates in large graphs
Abstract
Graphs play an increasingly important role in various big data applications. However, existing graph data structures cannot simultaneously address the performance bottlenecks caused by the dynamic updates, large scale, and high query complexity of current graphs. This paper proposes a novel data structure for large-scale dynamic graphs called CuckooGraph. It does not require any prior knowledge of the upcoming graphs, and can adaptively resize to the most memory-efficient form while requiring few memory accesses for very fast graph data processing. The key techniques of CuckooGraph include TRANSFORMATION and DENYLIST. TRANSFORMATION fully utilizes the limited memory by designing related data structures that allow flexible space transformations to smoothly expand/tighten the required space depending on the number of incoming items. DENYLIST efficiently handles item insertion failures and…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsGraph Theory and Algorithms · Algorithms and Data Compression · Opportunistic and Delay-Tolerant Networks
