TAPER: query-aware, partition-enhancement for large, heterogenous, graphs

TL;DR

TAPER is a system that adaptively refines graph partitioning to minimize query-related inter-partition traversals, improving scalability and responsiveness for large, heterogeneous graphs.

Contribution

TAPER introduces an iterative, workload-aware partition adjustment method that enhances existing algorithms for online, query-sensitive graph partitioning.

Findings

Reduces inter-partition traversals by 80% with hash-based partitioning.

Achieves 30% reduction with METIS partitioning.

Operates efficiently within 8 iterations.

Abstract

Graph partitioning has long been seen as a viable approach to address Graph DBMS scalability. A partitioning, however, may introduce extra query processing latency unless it is sensitive to a specific query workload, and optimised to minimise inter-partition traversals for that workload. Additionally, it should also be possible to incrementally adjust the partitioning in reaction to changes in the graph topology, the query workload, or both. Because of their complexity, current partitioning algorithms fall short of one or both of these requirements, as they are designed for offline use and as one-off operations. The TAPER system aims to address both requirements, whilst leveraging existing partitioning algorithms. TAPER takes any given initial partitioning as a starting point, and iteratively adjusts it by swapping chosen vertices across partitions, heuristically reducing the probability of inter-partition traversals for a given pattern matching queries workload. Iterations are inexpensive thanks to time and space optimisations in the underlying support data structures. We evaluate TAPER on two different large test graphs and over realistic query workloads. Our results indicate that, given a hash-based partitioning, TAPER reduces the number of inter-partition traversals by around 80%; given an unweighted METIS partitioning, by around 30%. These reductions are achieved within 8 iterations and with the additional advantage of being workload-aware and usable online.