Principles of Distributed Data Management in 2020?

TL;DR

This paper discusses the evolution of distributed data management principles, contrasting traditional generic techniques with emerging ad-hoc systems like MapReduce and Hadoop, and questions the need for new foundational models.

Contribution

It analyzes the fundamental principles behind new distributed data management solutions and explores whether a generic architectural model is needed for these emerging systems.

Findings

Emerging systems are often ad-hoc and may hinder data interoperability.

Fundamental principles of traditional distributed data management are still relevant.

The paper questions the adequacy of existing foundations for new data distribution solutions.

Abstract

With the advents of high-speed networks, fast commodity hardware, and the web, distributed data sources have become ubiquitous. The third edition of the \"Ozsu-Valduriez textbook Principles of Distributed Database Systems [10] reflects the evolution of distributed data management and distributed database systems. In this new edition, the fundamental principles of distributed data management could be still presented based on the three dimensions of earlier editions: distribution, heterogeneity and autonomy of the data sources. In retrospect, the focus on fundamental principles and generic techniques has been useful not only to understand and teach the material, but also to enable an infinite number of variations. The primary application of these generic techniques has been obviously for distributed and parallel DBMS versions. Today, to support the requirements of important data-intensive applications (e.g. social networks, web data analytics, scientific applications, etc.), new distributed data management techniques and systems (e.g. MapReduce, Hadoop, SciDB, Peanut, Pig latin, etc.) are emerging and receiving much attention from the research community. Although they do well in terms of consistency/flexibility/performance trade-offs for specific applications, they seem to be ad-hoc and might hurt data interoperability. The key questions I discuss are: What are the fundamental principles behind the emerging solutions? Is there any generic architectural model, to explain those principles? Do we need new foundations to look at data distribution?