Query-by-Sketch: Scaling Shortest Path Graph Queries on Very Large Networks

TL;DR

This paper introduces Query-by-Sketch, a scalable method for efficiently computing shortest path graphs in large networks, enabling rapid query responses even on billion-scale graphs.

Contribution

The paper presents Query-by-Sketch, a novel approach combining offline labeling and sketching to scale shortest path graph queries to extremely large networks.

Findings

QbS answers queries in microseconds for million-scale graphs.

QbS answers in less than half a second for billion-scale graphs.

Method is theoretically proven correct and empirically validated.

Abstract

Computing shortest paths is a fundamental operation in processing graph data. In many real-world applications, discovering shortest paths between two vertices empowers us to make full use of the underlying structure to understand how vertices are related in a graph, e.g. the strength of social ties between individuals in a social network. In this paper, we study the shortest-path-graph problem that aims to efficiently compute a shortest path graph containing exactly all shortest paths between any arbitrary pair of vertices on complex networks. Our goal is to design an exact solution that can scale to graphs with millions or billions of vertices and edges. To achieve high scalability, we propose a novel method, Query-by-Sketch (QbS), which efficiently leverages offline labelling (i.e., precomputed labels) to guide online searching through a fast sketching process that summarizes the important structural aspects of shortest paths in answering shortest-path-graph queries. We theoretically prove the correctness of this method and analyze its computational complexity. To empirically verify the efficiency of QbS, we conduct experiments on 12 real-world datasets, among which the largest dataset has 1.7 billion vertices and 7.8 billion edges. The experimental results show that QbS can answer shortest-path graph queries in microseconds for million-scale graphs and less than half a second for billion-scale graphs.