Compressed k2-Triples for Full-In-Memory RDF Engines

TL;DR

This paper introduces k2-triples, a highly compressed RDF data structure enabling full-in-memory SPARQL query processing on large datasets, significantly improving performance over existing methods.

Contribution

It proposes a novel compact k2-tree based RDF structure that achieves ultra-compression and supports in-memory query execution without decompression.

Findings

k2-triples outperform state-of-the-art compression methods

Supports full-in-memory SPARQL queries on large RDF datasets

Competitive with multi-index solutions in query performance

Abstract

Current "data deluge" has flooded the Web of Data with very large RDF datasets. They are hosted and queried through SPARQL endpoints which act as nodes of a semantic net built on the principles of the Linked Data project. Although this is a realistic philosophy for global data publishing, its query performance is diminished when the RDF engines (behind the endpoints) manage these huge datasets. Their indexes cannot be fully loaded in main memory, hence these systems need to perform slow disk accesses to solve SPARQL queries. This paper addresses this problem by a compact indexed RDF structure (called k2-triples) applying compact k2-tree structures to the well-known vertical-partitioning technique. It obtains an ultra-compressed representation of large RDF graphs and allows SPARQL queries to be full-in-memory performed without decompression. We show that k2-triples clearly outperforms state-of-the-art compressibility and traditional vertical-partitioning query resolution, remaining very competitive with multi-index solutions.