A Datalog-based Computational Model for Coordination-free, Data-Parallel Systems

TL;DR

This paper develops a Datalog-based computational model for data-parallel systems that are eventually consistent and coordination-free, extending CALM principles to synchronous, reliable communication settings.

Contribution

It introduces techniques to apply CALM principles to data-parallel systems, focusing on connected monotonic queries and analyzing the relationship with existing models.

Findings

CALM principle applies to connected monotonic queries in data-parallel systems.

Techniques developed can determine when coordination can be avoided.

The model subsumes previous asynchronous CALM models under relaxed synchronization constraints.

Abstract

Cloud computing refers to maximizing efficiency by sharing computational and storage resources, while data-parallel systems exploit the resources available in the cloud to perform parallel transformations over large amounts of data. In the same line, considerable emphasis has been recently given to two apparently disjoint research topics: data-parallel, and eventually consistent, distributed systems. Declarative networking has been recently proposed to ease the task of programming in the cloud, by allowing the programmer to express only the desired result and leave the implementation details to the responsibility of the run-time system. In this context, we propose a study on a logic-programming-based computational model for eventually consistent, data-parallel systems, the keystone of which is provided by the recent finding that the class of programs that can be computed in an eventually consistent, coordination-free way is that of monotonic programs. This principle is called CALM and has been proven by Ameloot et al. for distributed, asynchronous settings. We advocate that CALM should be employed as a basic theoretical tool also for data-parallel systems, wherein computation usually proceeds synchronously in rounds and where communication is assumed to be reliable. It is general opinion that coordination-freedom can be seen as a major discriminant factor. In this work we make the case that the current form of CALM does not hold in general for data-parallel systems, and show how, using novel techniques, the satisfiability of the CALM principle can still be obtained although just for the subclass of programs called connected monotonic queries. We complete the study with considerations on the relationships between our model and the one employed by Ameloot et al., showing that our techniques subsume the latter when the synchronization constraints imposed on the system are loosened.