Federated Data Science to Break Down Silos [Vision]

TL;DR

This paper proposes KEK, a federated platform for sharing, searching, and combining data science pipelines and artifacts across platforms, aiming to break down data silos and enhance collaboration.

Contribution

It introduces KEK, an open federated data science platform that enables sharing, semantic search, and pipeline integration across different systems.

Findings

KEK facilitates efficient search of related data science artifacts.

It supports combining pipelines across platforms in a federated manner.

Addresses the challenge of finding semantically related artifacts.

Abstract

Similar to Open Data initiatives, data science as a community has launched initiatives for sharing not only data but entire pipelines, derivatives, artifacts, etc. (Open Data Science). However, the few efforts that exist focus on the technical part on how to facilitate sharing, conversion, etc. This vision paper goes a step further and proposes KEK, an open federated data science platform that does not only allow for sharing data science pipelines and their (meta)data but also provides methods for efficient search and, in the ideal case, even allows for combining and defining pipelines across platforms in a federated manner. In doing so, KEK addresses the so far neglected challenge of actually finding artifacts that are semantically related and that can be combined to achieve a certain goal.