Distributed Subgraph Enumeration via Backtracking-based Framework

TL;DR

This paper introduces distributed subgraph enumeration frameworks, B-BENU and S-BENU, that improve efficiency by avoiding data shuffling and supporting dynamic graphs, significantly outperforming existing methods.

Contribution

The paper proposes novel distributed frameworks for subgraph enumeration that eliminate data shuffling and support incremental updates, enhancing scalability and efficiency.

Findings

B-BENU and S-BENU outperform state-of-the-art methods by up to 100x and 200x.

They effectively handle large data graphs and complex pattern graphs.

The frameworks support dynamic data graphs with incremental pattern enumeration.

Abstract

Finding or monitoring subgraph instances that are isomorphic to a given pattern graph in a data graph is a fundamental query operation in many graph analytic applications, such as network motif mining and fraud detection. The state-of-the-art distributed methods are inefficient in communication. They have to shuffle partial matching results during the distributed multiway join. The partial matching results may be much larger than the data graph itself. To overcome the drawback, we develop the Batch-BENU framework (B-BENU) for distributed subgraph enumeration. B-BENU executes a group of local search tasks in parallel. Each task enumerates subgraphs around a vertex in the data graph, guided by a backtracking-based execution plan. B-BENU does not shuffle any partial matching result. Instead, it stores the data graph in a distributed database. Each task queries adjacency sets of the data graph on demand. To support dynamic data graphs, we propose the concept of incremental pattern graphs and turn continuous subgraph enumeration into enumerating incremental pattern graphs at each time step. We develop the Streaming-BENU framework (S-BENU) to enumerate their matches efficiently. We implement B-BENU and S-BENU with the local database cache and the task splitting techniques. The extensive experiments show that B-BENU and S-BENU can scale to big data graphs and complex pattern graphs. They outperform the state-of-the-art methods by up to one and two orders of magnitude, respectively.