Learned Adaptive Indexing

TL;DR

This paper introduces a novel learned adaptive indexing method that builds indexes dynamically during query processing, leveraging workload prediction to improve performance in changing environments.

Contribution

It presents the first learned adaptive index that constructs itself on the fly, combining machine learning with adaptive indexing techniques for dynamic workloads.

Findings

Achieves 1.2x to 5.6x faster query performance

Outperforms existing adaptive indexes in most tested scenarios

Utilizes workload prediction to enhance index effectiveness

Abstract

Indexes can significantly improve search performance in relational databases. However, if the query workload changes frequently or new data updates occur continuously, it may not be worthwhile to build a conventional index upfront for query processing. Adaptive indexing is a technique in which an index gets built on the fly as a byproduct of query processing. In recent years, research in database indexing has taken a new direction where machine learning models are employed for the purpose of indexing. These indexes, known as learned indexes, can be more efficient compared to traditional indexes such as B+-tree in terms of memory footprints and query performance. However, a learned index has to be constructed upfront and requires training the model in advance, which becomes a challenge in dynamic situations when workload changes frequently. To the best of our knowledge, no learned indexes exist yet for adaptive indexing. We propose a novel learned approach for adaptive indexing. It is built on the fly as queries are submitted and utilizes learned models for indexing data. To enhance query performance, we employ a query workload prediction technique that makes future workload projection based on past workload data. We have evaluated our learned adaptive indexing approach against existing adaptive indexes for various query workloads. Our results show that our approach performs better than others in most cases, offering 1.2x - 5.6x improvement in query performance.