GeoFlink: A Distributed and Scalable Framework for the Real-time Processing of Spatial Streams

TL;DR

GeoFlink extends Apache Flink to efficiently process spatial data streams, supporting spatial queries with high throughput, filling a gap in scalable real-time spatial data processing.

Contribution

The paper introduces GeoFlink, a novel extension of Apache Flink that adds support for spatial data types, indexes, and continuous spatial queries in streaming environments.

Findings

GeoFlink achieves higher query throughput than standard Flink.

Supports spatial range, kNN, and join queries on point data.

Demonstrates effectiveness on real spatial data streams.

Abstract

Apache Flink is an open-source system for scalable processing of batch and streaming data. Flink does not natively support efficient processing of spatial data streams, which is a requirement of many applications dealing with spatial data. Besides Flink, other scalable spatial data processing platforms including GeoSpark, Spatial Hadoop, etc. do not support streaming workloads and can only handle static/batch workloads. To fill this gap, we present GeoFlink, which extends Apache Flink to support spatial data types, indexes and continuous queries over spatial data streams. To enable the efficient processing of spatial continuous queries and for the effective data distribution across Flink cluster nodes, a gird-based index is introduced. GeoFlink currently supports spatial range, spatial $k$NN and spatial join queries on point data type. An extensive experimental study on real spatial data streams shows that GeoFlink achieves significantly higher query throughput than ordinary Flink processing.