Benchmarking Data Lakes Featuring Structured and Unstructured Data with DLBench

TL;DR

DLBench is a comprehensive benchmark designed to evaluate and compare data lake systems supporting structured and unstructured data, addressing the lack of standardized evaluation criteria in this domain.

Contribution

The paper introduces DLBench, including a data model, workload model, and performance metrics, to standardize evaluation of data lake implementations.

Findings

DLBench effectively differentiates data lake systems based on performance.

The benchmark provides a unified framework for comparing textual and tabular data handling.

Evaluation of an open source data lake system demonstrates DLBench's practical utility.

Abstract

In the last few years, the concept of data lake has become trendy for data storage and analysis. Thus, several design alternatives have been proposed to build data lake systems. However, these proposals are difficult to evaluate as there are no commonly shared criteria for comparing data lake systems. Thus, we introduce DLBench, a benchmark to evaluate and compare data lake implementations that support textual and/or tabular contents. More concretely, we propose a data model made of both textual and raw tabular documents, a workload model composed of a set of various tasks, as well as a set of performance-based metrics, all relevant to the context of data lakes. As a proof of concept, we use DLBench to evaluate an open source data lake system we previously developed.