FlexFlood: Efficiently Updatable Learned Multi-dimensional Index

TL;DR

FlexFlood is a novel learned multi-dimensional index that maintains efficient query performance under skewed data distributions by partial reconstruction, guaranteeing update time complexity and outperforming existing methods in speed.

Contribution

It introduces FlexFlood, the first learned multi-dimensional index with guaranteed update time complexity and adaptive partial reconstruction for skewed data distributions.

Findings

Search performance up to 10 times faster on skewed data.

Partial reconstruction takes about twice as long as naive updates.

FlexFlood outperforms existing methods in real-world and artificial datasets.

Abstract

A learned multi-dimensional index is a data structure that efficiently answers multi-dimensional orthogonal queries by understanding the data distribution using machine learning models. One of the existing problems is that the search performance significantly decreases when the distribution of data stored in the data structure becomes skewed due to update operations. To overcome this problem, we propose FlexFlood, a flexible variant of Flood. FlexFlood partially reconstructs the internal structure when the data distribution becomes skewed. Moreover, FlexFlood is the first learned multi-dimensional index that guarantees the time complexity of the update operation. Through experiments using both artificial and real-world data, we demonstrate that the search performance when the data distribution becomes skewed is up to 10 times faster than existing methods. We also found that partial reconstruction takes only about twice as much time as naive data updating.