Adaptive Partitioning and its Applicability to a Highly Scalable and Available Geo-Spatial Indexing Solution

TL;DR

This paper introduces Adaptive Partitioning, a scalable and highly available geo-spatial indexing method designed to efficiently handle real-time satellite tracking data and complex spatial queries.

Contribution

It presents a novel adaptive partitioning approach tailored for high-volume, real-time geo-spatial data indexing and querying in satellite tracking systems.

Findings

Supports real-time data ingestion and complex spatial queries

Provides high availability and scalability

Reduces costs compared to traditional methods

Abstract

Satellite Tracking of People (STOP) tracks thousands of GPS-enabled devices 24 hours a day and 365 days a year. With locations captured for each device every minute, STOP servers receive tens of millions of points each day. In addition to cataloging these points in real-time, STOP must also respond to questions from customers such as, "What devices of mine were at this location two months ago?" They often then broaden their question to one such as, "Which of my devices have ever been at this location?" The processing requirements necessary to answer these questions while continuing to process inbound data in real-time is non-trivial. To meet this demand, STOP developed Adaptive Partitioning to provide a cost-effective and highly available hardware platform for the geographical and time-spatial indexing capabilities necessary for responding to customer data requests while continuing to catalog inbound data in real-time.