One Size Does NOT Fit All: On the Importance of Physical Representations for Datalog Evaluation

TL;DR

This paper emphasizes the importance of selecting appropriate physical data representations in Datalog engines, demonstrating through experiments that workload-specific choices significantly impact performance and proposing an automatic selection method.

Contribution

It introduces an in-depth experimental analysis of physical representations in Datalog evaluation and develops a decision tree-based mechanism for workload-aware selection.

Findings

Physical representation choice greatly affects Datalog performance.

Workload characteristics such as relation size and recursiveness influence optimal representation.

The proposed decision tree approach effectively automates representation selection.

Abstract

Datalog is an increasingly popular recursive query language that is declarative by design, meaning its programs must be translated by an engine into the actual physical execution plan. When generating this plan, a central decision is how to physically represent all involved relations, an aspect in which existing Datalog engines are surprisingly restrictive and often resort to one-size-fits-all solutions. The reason for this is that the typical execution plan of a Datalog program not only performs a single type of operation against the physical representations, but a mixture of operations, such as insertions, lookups, and containment-checks. Further, the relevance of each operation type highly depends on the workload characteristics, which range from familiar properties such as the size, multiplicity, and arity of the individual relations to very specific Datalog properties, such as the "interweaving" of rules when relations occur multiple times, and in particular the recursiveness of the query which might generate new tuples on the fly during evaluation. This indicates that a variety of physical representations, each with its own strengths and weaknesses, is required to meet the specific needs of different workload situations. To evaluate this, we conduct an in-depth experimental study of the interplay between potentially suitable physical representations and seven dimensions of workload characteristics that vary across actual Datalog programs, revealing which properties actually matter. Based on these insights, we design an automatic selection mechanism that utilizes a set of decision trees to identify suitable physical representations for a given workload.