BuffCut: Prioritized Buffered Streaming Graph Partitioning

TL;DR

BuffCut is a buffered streaming graph partitioning method that improves partition quality and efficiency by combining prioritized buffering with batch-wise multilevel assignment, especially under adversarial stream order.

Contribution

It introduces a novel prioritized buffering approach that enhances streaming graph partitioning quality and efficiency over existing methods.

Findings

BuffCut reduces edge cuts by 20.8% compared to state-of-the-art buffered methods.

It runs 2.9 times faster and uses 11.3 times less memory than the strongest baseline.

BuffCut outperforms previous methods across diverse real-world and synthetic graphs.

Abstract

Streaming graph partitioners enable resource-efficient and massively scalable partitioning, but one-pass assignment heuristics are highly sensitive to stream order and often yield substantially higher edge cuts than in-memory methods. We present BuffCut, a buffered streaming partitioner that narrows this quality gap, particularly when stream ordering is adversarial, by combining prioritized buffering with batch-wise multilevel assignment. BuffCut maintains a bounded priority buffer to delay poorly informed decisions and regulate the order in which nodes are considered for assignment. It incrementally constructs high-locality batches of configurable size by iteratively inserting the highest-priority nodes from the buffer into the batch, effectively recovering locality structure from the stream. Each batch is then assigned via a multilevel partitioning algorithm. Experiments on diverse real-world and synthetic graphs show that BuffCut consistently outperforms state-of-the-art buffered streaming methods. Compared to the strongest prioritized buffering baseline, BuffCut achieves 20.8% fewer edge cuts while running 2.9 times faster and using 11.3 times less memory. Against the next-best buffered method, it reduces edge cut by 15.8% with only modest overheads of 1.8 times runtime and 1.09 times memory.