MEDAL: An AI-driven Data Fabric Concept for Elastic Cloud-to-Edge Intelligence

TL;DR

MEDAL introduces an AI-driven data fabric that enhances data management and orchestration across cloud and edge environments, enabling efficient data-centric applications and workflows.

Contribution

It presents a novel intelligent data fabric architecture supporting end-to-end data operations across cloud and edge, with automation and federation capabilities.

Findings

Supports seamless data workflow management across Cloud and Edge.

Automates data operations and resource orchestration.

Demonstrated with a connected cars use case.

Abstract

Current Cloud solutions for Edge Computing are inefficient for data-centric applications, as they focus on the IaaS/PaaS level and they miss the data modeling and operations perspective. Consequently, Edge Computing opportunities are lost due to cumbersome and data assets-agnostic processes for end-to-end deployment over the Cloud-to-Edge continuum. In this paper, we introduce MEDAL, an intelligent Cloud-to-Edge Data Fabric to support Data Operations (DataOps)across the continuum and to automate management and orchestration operations over a combined view of the data and the resource layer. MEDAL facilitates building and managing data workflows on top of existing flexible and composable data services, seamlessly exploiting and federating IaaS/PaaS/SaaS resources across different Cloud and Edge environments. We describe the MEDAL Platform as a usable tool for Data Scientists and Engineers, encompassing our concept and we illustrate its application though a connected cars use case.