Using Learned Indexes to Improve Time Series Indexing Performance on Embedded Sensor Devices

TL;DR

This paper explores the use of learned indexes to significantly enhance time series data querying efficiency on resource-constrained embedded sensor devices, reducing I/O and increasing throughput.

Contribution

It introduces learned index structures tailored for embedded devices, demonstrating substantial performance improvements over existing index algorithms.

Findings

Query I/O reduced by 50-90%

Query throughput improved by 2-5 times

Requires only a few kilobytes of RAM

Abstract

Efficiently querying data on embedded sensor and IoT devices is challenging given the very limited memory and CPU resources. With the increasing volumes of collected data, it is critical to process, filter, and manipulate data on the edge devices where it is collected to improve efficiency and reduce network transmissions. Existing embedded index structures do not adapt to the data distribution and characteristics. This paper demonstrates how applying learned indexes that develop space efficient summaries of the data can dramatically improve the query performance and predictability. Learned indexes based on linear approximations can reduce the query I/O by 50 to 90% and improve query throughput by a factor of 2 to 5, while only requiring a few kilobytes of RAM. Experimental results on a variety of time series data sets demonstrate the advantages of learned indexes that considerably improve over the state-of-the-art index algorithms.