Q-Graph: Preserving Query Locality in Multi-Query Graph Processing

TL;DR

Q-Graph is a system designed to optimize multi-query graph processing by considering query locality, dynamically adapting partitioning and synchronization to reduce latency in user-centric applications.

Contribution

It introduces query-aware partitioning and synchronization methods that adapt at runtime, significantly improving performance over traditional static approaches.

Findings

Q-cut reduces average query latency by up to 57%.

Dynamic adaptation maintains high locality under changing workloads.

The system effectively exploits query hotspots for efficiency.

Abstract

Arising user-centric graph applications such as route planning and personalized social network analysis have initiated a shift of paradigms in modern graph processing systems towards multi-query analysis, i.e., processing multiple graph queries in parallel on a shared graph. These applications generate a dynamic number of localized queries around query hotspots such as popular urban areas. However, existing graph processing systems are not yet tailored towards these properties: The employed methods for graph partitioning and synchronization management disregard query locality and dynamism which leads to high query latency. To this end, we propose the system Q-Graph for multi-query graph analysis that considers query locality on three levels. (i) The query-aware graph partitioning algorithm Q-cut maximizes query locality to reduce communication overhead. (ii) The method for synchronization management, called hybrid barrier synchronization, allows for full exploitation of local queries spanning only a subset of partitions. (iii) Both methods adapt at runtime to changing query workloads in order to maintain and exploit locality. Our experiments show that Q-cut reduces average query latency by up to 57 percent compared to static query-agnostic partitioning algorithms.