LiveData -- A Worldwide Data Mesh for Stratified Data

TL;DR

LiveData is a novel worldwide data mesh architecture that transforms data heterogeneity into a feature, enabling high-quality, diverse, and interoperable data sharing across countries to facilitate data reuse and integration.

Contribution

The paper introduces LiveData, a new data distribution network architecture that explicitly models data heterogeneity as a feature to improve cross-country data reuse.

Findings

Specification of LiveData architecture and data handling

Implementation of a use case between Italy and Mongolia

Enhanced data diversity and interoperability

Abstract

Data reuse is fundamental for reducing the data integration effort required to build data supporting new applications, especially in data scarcity contexts. However, data reuse requires to deal with data heterogeneity, which is always present in data coming from different sources. Such heterogeneity appears at different levels, like the language used by the data, the structure of the information it represents, and the data types and formats adopted by the datasets. Despite the valuable insights gained by reusing data across contexts, dealing with data heterogeneity is still a high price to pay. Additionally, data reuse is hampered by the lack of data distribution infrastructures supporting the production and distribution of quality and interoperable data. These issues affecting data reuse are amplified considering cross-country data reuse, where geographical and cultural differences are more pronounced. In this paper, we propose LiveData, a cross-country data distribution network handling high quality and diversity-aware data. LiveData is composed by different nodes having an architecture providing components for the generation and distribution of a new type of data, where heterogeneity is transformed into information diversity and considered as a feature, explicitly defined and used to satisfy the data users purposes. This paper presents the specification of the LiveData network, by defining the architecture and the type of data handled by its nodes. This specification is currently being used to implement a concrete use case for data reuse and integration between the University of Trento (Italy) and the National University of Mongolia.