In-database connected component analysis

TL;DR

This paper presents a practical, SQL-compatible algorithm for efficiently computing connected components in large graph data within a parallel relational database, ensuring correctness and probabilistic efficiency.

Contribution

It introduces a linear-space, randomized algorithm for connected components that guarantees correctness and achieves near-linear runtime with high probability in a parallel database setting.

Findings

Algorithm terminates after O(log |V|) SQL queries with high probability

Empirical results show quasi-linear runtime performance

Algorithm is suitable for large-scale graph data in MPP relational databases

Abstract

We describe a Big Data-practical, SQL-implementable algorithm for efficiently determining connected components for graph data stored in a Massively Parallel Processing (MPP) relational database. The algorithm described is a linear-space, randomised algorithm, always terminating with the correct answer but subject to a stochastic running time, such that for any $\epsilon>0$ and any input graph $G=\langle V, E \rangle$ the algorithm terminates after $\mathop{\text{O}}(\log |V|)$ SQL queries with probability of at least $1-\epsilon$, which we show empirically to translate to a quasi-linear runtime in practice.