Scaling Up Knowledge Graph Creation to Large and Heterogeneous Data Sources

TL;DR

This paper introduces an optimization framework for automating and improving the efficiency of large-scale, heterogeneous knowledge graph creation from diverse data sources using declarative planning and execution strategies.

Contribution

It proposes a novel planning and transformation approach that partitions and schedules RML mapping assertions, significantly enhancing performance in complex KG creation tasks.

Findings

Performance improvements in RML engines with large data sources

Enables partial KG generation in previously timeout-prone scenarios

Effective optimization of assertion execution order and partitioning

Abstract

RDF knowledge graphs (KG) are powerful data structures to represent factual statements created from heterogeneous data sources. KG creation is laborious and demands data management techniques to be executed efficiently. This paper tackles the problem of the automatic generation of KG creation processes declaratively specified; it proposes techniques for planning and transforming heterogeneous data into RDF triples following mapping assertions specified in the RDF Mapping Language (RML). Given a set of mapping assertions, the planner provides an optimized execution plan by partitioning and scheduling the execution of the assertions. First, the planner assesses an optimized number of partitions considering the number of data sources, type of mapping assertions, and the associations between different assertions. After providing a list of partitions and assertions that belong to each partition, the planner determines their execution order. A greedy algorithm is implemented to generate the partitions' bushy tree execution plan. Bushy tree plans are translated into operating system commands that guide the execution of the partitions of the mapping assertions in the order indicated by the bushy tree. The proposed optimization approach is evaluated over state-of-the-art RML-compliant engines, and existing benchmarks of data sources and RML triples maps. Our experimental results suggest that the performance of the studied engines can be considerably improved, particularly in a complex setting with numerous triples maps and large data sources. As a result, engines that time out in complex cases are enabled to produce at least a portion of the KG applying the planner.