Fixpoint Semantics and Optimization of Recursive Datalog Programs with Aggregates

TL;DR

This paper extends the fixpoint semantics and optimization techniques of Datalog to include recursive rules with aggregates like min, max, count, and sum, enhancing performance and scalability.

Contribution

It introduces a comprehensive solution for integrating aggregates into recursive Datalog programs while preserving their declarative semantics and improving efficiency.

Findings

Identifies classes of programs where min/max do not affect semantics.

Revisits monotonic count and sum, ensuring they preserve fixpoint semantics.

Combines monotonic aggregates with max to optimize recursive computations.

Abstract

A very desirable Datalog extension investigated by many researchers in the last thirty years consists in allowing the use of the basic SQL aggregates min, max, count and sum in recursive rules. In this paper, we propose a simple comprehensive solution that extends the declarative least-fixpoint semantics of Horn Clauses, along with the optimization techniques used in the bottom-up implementation approach adopted by many Datalog systems. We start by identifying a large class of programs of great practical interest in which the use of min or max in recursive rules does not compromise the declarative fixpoint semantics of the programs using those rules. Then, we revisit the monotonic versions of count and sum aggregates proposed in (Mazuran et al. 2013b) and named, respectively, mcount and msum. Since mcount, and also msum on positive numbers, are monotonic in the lattice of set-containment, they preserve the fixpoint semantics of Horn Clauses. However, in many applications of practical interest, their use can lead to inefficiencies, that can be eliminated by combining them with max, whereby mcount and msum become the standard count and sum. Therefore, the semantics and optimization techniques of Datalog are extended to recursive programs with min, max, count and sum, making possible the advanced applications of superior performance and scalability demonstrated by BigDatalog (Shkapsky et al. 2016) and Datalog-MC (Yang et al. 2017). This paper is under consideration for acceptance in TPLP.