Finding Your Way Through the Jungle of Big Data Architectures

TL;DR

This paper systematically reviews various big data architectures, comparing their features and interdependencies, and proposes a framework to guide selecting appropriate architectures for specific data management needs.

Contribution

It offers a first comprehensive view of modern data architectures like Data Mesh and Data Fabric, and suggests a pattern-based approach for architecture selection.

Findings

Provides a comparative analysis of data architectures

Identifies interdependencies among architectures

Proposes a pattern system for architecture selection

Abstract

This paper presents a systematic review of common analytical data architectures based on DAMA-DMBOK and ArchiMate. The paper is work in progress and provides a first view on Gartner's Logical Data Warehouse paradigm, Data Fabric and Dehghani's Data Mesh proposal as well as their interdependencies. It furthermore sketches the way forward how this work can be extended by covering more architecture paradigms (incl. classic Data Warehouse, Data Vault, Data Lake, Lambda and Kappa architectures) and introducing a template with among others "context", "problem" and "solution" descriptions, leading ultimately to a pattern system providing guidance for choosing the right architecture paradigm for the right situation.