General-Purpose Join Algorithms for Listing Triangles in Large Graphs

TL;DR

This paper explores applying a worst-case optimal join algorithm, Leapfrog Triejoin, with a novel data feeding approach called boxing, to efficiently list triangles in large graphs, matching specialized algorithms in I/O complexity.

Contribution

It introduces 'boxing', a simple method for applying LFTJ to large data, and demonstrates its efficiency and optimality in out-of-core triangle listing, with theoretical and experimental validation.

Findings

Boxed LFTJ is I/O efficient and worst-case optimal.

Boxed LFTJ matches specialized algorithms like MGT in complexity.

Experimental results show LFTJ is within a factor of 3 of MGT on large datasets.

Abstract

We investigate applying general-purpose join algorithms to the triangle listing problem in an out-of-core context. In particular, we focus on Leapfrog Triejoin (LFTJ) by Veldhuizen 2014, a recently proposed, worst-case optimal algorithm. We present "boxing": a novel, yet conceptually simple, approach for feeding input data to LFTJ. Our extensive analysis shows that this approach is I/O efficient, being worst-case optimal (in a certain sense). Furthermore, if input data is only a constant factor larger than the available memory, then a boxed LFTJ essentially maintains the CPU data-complexity of the vanilla LFTJ. Next, focusing on LFTJ applied to the triangle query, we show that for many graphs boxed LFTJ matches the I/O complexity of the recently by Hu, Tao and Yufei proposed specialized algorithm MGT for listing tiangles in an out-of-core setting. We also strengthen the analysis of LFTJ's computational complexity for the triangle query by considering families of input graphs that are characterized not only by the number of edges but also by a measure of their density. E.g., we show that LFTJ achieves a CPU complexity of O(|E|log|E|) for planar graphs, while on general graphs, no algorithm can be faster than O(|E|^{1.5}). Finally, we perform an experimental evaluation for the triangle listing problem confirming our theoretical results and showing the overall effectiveness of our approach. On all our real-world and synthetic data sets (some of which containing more than 1.2 billion edges) LFTJ in single-threaded mode is within a factor of 3 of the specialized MGT; a penalty that---as we demonstrate---can be alleviated by parallelization.