Distributed Spatial-Keyword kNN Monitoring for Location-aware Pub/Sub

TL;DR

This paper presents DkM-SKS, a distributed system for real-time spatial-keyword kNN monitoring in large-scale Pub/Sub environments, balancing load and improving update efficiency.

Contribution

It introduces a scalable, distributed solution for spatial-keyword kNN monitoring that balances load among workers and accelerates updates using in-memory indexing.

Findings

Demonstrates efficiency and scalability through experiments on real datasets.

Achieves load balancing across distributed workers.

Improves update speed with in-memory indexing.

Abstract

Recent applications employ publish/subscribe (Pub/Sub) systems so that publishers can easily receive attentions of customers and subscribers can monitor useful information generated by publishers. Due to the prevalence of smart devices and social networking services, a large number of objects that contain both spatial and keyword information have been generated continuously, and the number of subscribers also continues to increase. This poses a challenge to Pub/Sub systems: they need to continuously extract useful information from massive objects for each subscriber in real time. In this paper, we address the problem of k nearest neighbor monitoring on a spatial-keyword data stream for a large number of subscriptions. To scale well to massive objects and subscriptions, we propose a distributed solution, namely DkM-SKS. Given m workers, DkM-SKS divides a set of subscriptions into m disjoint subsets based on a cost model so that each worker has almost the same kNN-update cost, to maintain load balancing. DkM-SKS allows an arbitrary approach to updating kNN of each subscription, so with a suitable in-memory index, DkM-SKS can accelerate update efficiency by pruning irrelevant subscriptions for a given new object. We conduct experiments on real datasets, and the results demonstrate the efficiency and scalability of DkM-SKS.