GraphLake: A Purpose-Built Graph Compute Engine for Lakehouse

TL;DR

GraphLake is a specialized graph compute engine designed for Lakehouse architectures, leveraging graph-aware techniques to enable efficient graph analytics directly over Lakehouse tables with improved performance.

Contribution

It introduces GraphLake, a novel engine that maps Lakehouse tables to graph structures and optimizes query performance with new techniques, outperforming existing solutions.

Findings

Significantly lower startup time compared to PuppyGraph

Enhanced query efficiency through graph-aware caching and parallel primitives

Demonstrated superior performance in extensive experiments

Abstract

In this paper, we introduce GraphLake, a purpose-built graph compute engine for Lakehouse. GraphLake is built on top of the commercial graph database TigerGraph. It maps Lakehouse tables to vertex and edge types in a labeled property graph and supports graph analytics over Lakehouse tables using GSQL. To minimize startup time, it loads only the graph topology. Furthermore, it introduces a series of techniques to ensure query efficiency over Lakehouse tables, including a graph-aware caching mechanism and two Lakehouse-optimized parallel primitives. Extensive experiments demonstrate that GraphLake significantly outperforms PuppyGraph, the current state-of-the-art graph compute engine for Lakehouse, by achieving both lower startup and query time.