Ten Pillars for Data Meshes

TL;DR

This paper introduces ten foundational principles, called pillars, for constructing and integrating data meshes from multiple data platforms to enhance data interoperability, accelerate research, and reduce development costs.

Contribution

It proposes a set of ten principles to standardize and simplify the creation and joining of data meshes across diverse data platforms.

Findings

Ten principles for data mesh implementation

Enhanced data interoperability across platforms

Reduced costs and increased speed in data mesh deployment

Abstract

Over the past few years, a growing number of data platforms have emerged, including data commons, data repositories, and databases containing biomedical, environmental, social determinants of health and other data relevant to improving health outcomes. With the growing number of data platforms, interoperating multiple data platforms to form data meshes, data fabrics and other types of data ecosystems reduces data silos, expands data use, and increases the potential for new discoveries. In this paper, we introduce ten principles, which we call pillars, for data meshes. The goals of the principles are 1) to make it easier, faster, and more uniform to set up a data mesh from multiple data platforms; and, 2) to make it easier, faster, and more uniform, for a data platform to join one or more data meshes. The hope is that the greater availability of data through data meshes will accelerate research and that the greater uniformity of meshes will lower the cost of developing meshes and connecting a data platform to them.