Parallel In-Memory Evaluation of Spatial Joins

TL;DR

This paper explores the parallel in-memory evaluation of spatial joins, redesigning classic algorithms for modern hardware to significantly improve performance and scalability for large datasets.

Contribution

It introduces a re-designed partitioning-based algorithm optimized for parallel in-memory execution and provides a method to select parameters based on data statistics.

Findings

Performance improved significantly with tuning

Algorithm scales well with multiple threads

Join processing time reduced to under one second for large datasets

Abstract

The spatial join is a popular operation in spatial database systems and its evaluation is a well-studied problem. As main memories become bigger and faster and commodity hardware supports parallel processing, there is a need to revamp classic join algorithms which have been designed for I/O-bound processing. In view of this, we study the in-memory and parallel evaluation of spatial joins, by re-designing a classic partitioning-based algorithm to consider alternative approaches for space partitioning. Our study shows that, compared to a straightforward implementation of the algorithm, our tuning can improve performance significantly. We also show how to select appropriate partitioning parameters based on data statistics, in order to tune the algorithm for the given join inputs. Our parallel implementation scales gracefully with the number of threads reducing the cost of the join to at most one second even for join inputs with tens of millions of rectangles.