Querying Evolving Graphs with Portal

TL;DR

This paper introduces TGraph, a logical model for evolving graphs, and a temporal graph algebra TGA, implemented in Portal on Spark/GraphX, enabling scalable analysis of dynamic graph phenomena.

Contribution

It proposes a formal model and algebra for evolving graphs, and demonstrates a scalable implementation in Portal for real-world datasets.

Findings

Portal scales well on large datasets

TGA can express a wide range of evolving graph queries

The model captures both topology and attribute evolution

Abstract

Graphs are used to represent a plethora of phenomena, from the Web and social networks, to biological pathways, to semantic knowledge bases. Arguably the most interesting and important questions one can ask about graphs have to do with their evolution. Which Web pages are showing an increasing popularity trend? How does influence propagate in social networks? How does knowledge evolve? This paper proposes a logical model of an evolving graph called a TGraph, which captures evolution of graph topology and of its vertex and edge attributes. We present a compositional temporal graph algebra TGA, and show a reduction of TGA to temporal relational algebra with graph-specific primitives. We formally study the properties of TGA, and also show that it is sufficient to concisely express a wide range of common use cases. We describe an implementation of our model and algebra in Portal, built on top of Apache Spark / GraphX. We conduct extensive experiments on real datasets, and show that Portal scales.