A Unified Approach for Multi-Granularity Search over Spatial Datasets

TL;DR

This paper introduces Spadas, a unified multi-granularity spatial data search system that efficiently supports both dataset and data point searches through a novel index and pruning techniques, significantly outperforming existing methods.

Contribution

The paper presents a novel unified index and pruning mechanisms for multi-granularity spatial data search, enabling efficient and integrated dataset and data point queries.

Findings

Orders of magnitude faster than state-of-the-art algorithms

Effective filtering of non-relevant datasets and points

Successful implementation of an online spatial data search system

Abstract

There has been increased interest in data search as a means to find relevant datasets or data points in data lakes and repositories. Although approaches have been proposed to support spatial dataset search and data point search, they consider the two types of searches independently. To enable search operations ranging from the coarse-grained dataset level to the fine-grained data point level, we provide an integrated one that supports diverse query types and distance metrics. In this paper, we focus on designing a multi-granularity spatial data search system, called Spadas, that supports both dataset and data point search operations. To address the challenges of the high cost of indexing and susceptibility to outliers, we propose a unified index that can drastically improve query efficiency in various scenarios by organizing data reasonably and removing outliers in datasets. Moreover, to accelerate all data search operations, we propose a set of pruning mechanisms based on the unified index, including fast bound estimation, approximation technique with error bound, and pruning in batch techniques, to effectively filter out non-relevant datasets and points. Finally, we report the results of a detailed experimental evaluation using six spatial data repositories, achieving orders of magnitude faster than the state-of-the-art algorithms and demonstrating the effectiveness by case study. An online spatial data search system of Spadas is also implemented and made accessible to users.