A Critical Analysis of Recursive Model Indexes

TL;DR

This paper provides a broad analysis of recursive model indexes (RMIs), identifying key hyperparameters affecting performance, and offers a simple configuration guideline that achieves competitive results with less tuning effort.

Contribution

It is the first broad, inventor-independent analysis of RMIs, revealing the importance of hyperparameters and providing a practical guideline for effective configuration.

Findings

Hyperparameters like model type, layer size, error bounds, and search algorithms significantly impact RMI performance.

The proposed guideline achieves performance comparable to state-of-the-art RMIs with less tuning.

Reimplementation of RMIs improves build time by up to 6.3 times.

Abstract

The recursive model index (RMI) has recently been introduced as a machine-learned replacement for traditional indexes over sorted data, achieving remarkably fast lookups. Follow-up work focused on explaining RMI's performance and automatically configuring RMIs through enumeration. Unfortunately, configuring RMIs involves setting several hyperparameters, the enumeration of which is often too time-consuming in practice. Therefore, in this work, we conduct the first inventor-independent broad analysis of RMIs with the goal of understanding the impact of each hyperparameter on performance. In particular, we show that in addition to model types and layer size, error bounds and search algorithms must be considered to achieve the best possible performance. Based on our findings, we develop a simple-to-follow guideline for configuring RMIs. We evaluate our guideline by comparing the resulting RMIs with a number of state-of-the-art indexes, both learned and traditional. We show that our simple guideline is sufficient to achieve competitive performance with other learned indexes and RMIs whose configuration was determined using an expensive enumeration procedure. In addition, while carefully reimplementing RMIs, we are able to improve the build time by 2.5x to 6.3x.