Towards a Smart Data Processing and Storage Model

TL;DR

This paper discusses the design of a data processing and storage system that ensures traceability, consistency, and trustworthiness of data throughout its lifecycle, including when data is combined or transformed.

Contribution

It introduces a theoretical framework and architecture for a system supporting traceable and reliable data management, along with a prototype implementation in Pharo.

Findings

Identified key requirements for traceable data storage

Proposed a system architecture supporting data provenance

Developed a prototype demonstrating the feasibility

Abstract

In several domains it is crucial to store and manipulate data whose origin needs to be completely traceable to guarantee the consistency, trustworthiness and reliability on the data itself typically for ethical and legal reasons. It is also important to guarantee that such properties are also carried further when such data is composed and processed into new data. In this article we present the main requirements and theorethical problems that arise by the design of a system supporting data with such capabilities. We present an architecture for implementing a system as well as a prototype developed in Pharo.