Towards Adaptive Storage Views in Virtual Memory

TL;DR

This paper proposes an innovative approach to integrating adaptive, coarse-granular indexing directly into the storage layer of a DBMS by leveraging OS virtual memory features, aiming to streamline query processing and improve scan performance.

Contribution

It introduces a novel design that fuses storage and indexing layers using virtual memory views, created adaptively during query processing, reducing indirection and enhancing efficiency.

Findings

Adaptive virtual memory views improve query speed.

Automatic routing to optimal views enhances performance.

Gradual indexing leads to better scan efficiency.

Abstract

Traditionally, DBMSs separate their storage layer from their indexing layer. While the storage layer physically materializes the database and provides low-level access methods to it, the indexing layer on top enables a faster locating of searched-for entries. While this clearly separates concerns, it also adds a level of indirection to the already complex execution path. In this work, we propose an alternative design: Instead of conservatively separating both layers, we naturally fuse them by integrating an adaptive coarse-granular indexing scheme directly into the storage layer. We do so by utilizing tools of the virtual memory management subsystem provided by the OS: On the lowest level, we materialize the database content in form of physical main memory. On top of that, we allow the creation of arbitrarily many virtual memory storage views that map to subsets of the database having certain properties of interest. This creation happens fully adaptively as a side-product of query processing. To speed up query answering, we route each query automatically to the most fitting virtual view(s). By this, we naturally index the storage layer in its core and gradually improve the provided scan performance.