SOSD: A Benchmark for Learned Indexes

TL;DR

This paper introduces SOSD, a benchmarking framework for learned index structures, demonstrating that they often outperform traditional indexes on real-world datasets, thus validating their potential for data management systems.

Contribution

The paper presents SOSD, a comprehensive benchmark for learned indexes, including datasets and baselines, to evaluate their performance against traditional index structures.

Findings

Learned indexes often outperform traditional indexes on real data.

SOSD provides a standardized platform for evaluating learned index models.

Preliminary results support the promise of learned indexes for future systems.

Abstract

A groundswell of recent work has focused on improving data management systems with learned components. Specifically, work on learned index structures has proposed replacing traditional index structures, such as B-trees, with learned models. Given the decades of research committed to improving index structures, there is significant skepticism about whether learned indexes actually outperform state-of-the-art implementations of traditional structures on real-world data. To answer this question, we propose a new benchmarking framework that comes with a variety of real-world datasets and baseline implementations to compare against. We also show preliminary results for selected index structures, and find that learned models indeed often outperform state-of-the-art implementations, and are therefore a promising direction for future research.