Optimizing Federated Queries Based on the Physical Design of a Data Lake

TL;DR

This paper introduces heuristics for optimizing federated queries in Semantic Data Lakes by leveraging physical design information, leading to faster query processing.

Contribution

It presents a novel approach that incorporates physical design heuristics into query optimization for Semantic Data Lakes, improving efficiency.

Findings

Heuristics based on physical design significantly reduce query execution time.

Implementation on Ontario demonstrates practical improvements.

Exploiting index and normalization knowledge enhances query plan quality.

Abstract

The optimization of query execution plans is known to be crucial for reducing the query execution time. In particular, query optimization has been studied thoroughly for relational databases over the past decades. Recently, the Resource Description Framework (RDF) became popular for publishing data on the Web. As a consequence, federations composed of different data models like RDF and relational databases evolved. One type of these federations are Semantic Data Lakes where every data source is kept in its original data model and semantically annotated with ontologies or controlled vocabularies. However, state-of-the-art query engines for federated query processing over Semantic Data Lakes often rely on optimization techniques tailored for RDF. In this paper, we present query optimization techniques guided by heuristics that take the physical design of a Data Lake into account. The heuristics are implemented on top of Ontario, a SPARQL query engine for Semantic Data Lakes. Using source-specific heuristics, the query engine is able to generate more efficient query execution plans by exploiting the knowledge about indexes and normalization in relational databases. We show that heuristics which take the physical design of the Data Lake into account are able to speed up query processing.