RMLStreamer-SISO: an RDF stream generator from streaming heterogeneous data

TL;DR

This paper presents RMLStreamer-SISO, a scalable RDF stream generator from heterogeneous data streams that achieves low latency and high throughput, enabling efficient integration with stream reasoning systems.

Contribution

It extends RMLStreamer to generate RDF streams from dynamic heterogeneous data streams using a dynamic window approach, improving scalability and performance.

Findings

Achieves millisecond latency compared to seconds of previous solutions.

Maintains constant memory usage across workloads.

Sustains throughput of around 70,000 records/sec.

Abstract

Stream-reasoning query languages such as CQELS and C-SPARQL enable query answering over RDF streams. Unfortunately, there currently is a lack of efficient RDF stream generators to feed RDF stream reasoners. State-of-the-art RDF stream generators are limited with regard to the velocity and volume of streaming data they can handle. To efficiently generate RDF streams in a scalable way, we extended the RMLStreamer to also generate RDF streams from dynamic heterogeneous data streams. This paper introduces a scalable solution that relies on a dynamic window approach to generate RDF streams with low latency and high throughput from multiple heterogeneous data streams. Our evaluation shows that our solution outperforms the state-of-the-art by achieving millisecond latency (compared to seconds that state-of-the-art solutions need), constant memory usage for all workloads, and sustainable throughput of around 70,000 records/s (compared to 10,000 records/s that state-of-the-art solutions take). This opens up the access to numerous data streams for integration with the semantic web.