Optimal Hierarchical Layouts for Cache-Oblivious Search Trees

TL;DR

This paper introduces Hierarchical Layouts, a unified framework for cache-oblivious binary search tree layouts, and proposes the MinWEP layout that significantly reduces cache misses and improves performance.

Contribution

It unifies previous tree layout methods into a general framework and introduces the MinWEP layout optimized for cache efficiency.

Findings

MinWEP layout outperforms previous layouts by nearly 20%

Hierarchical Layouts generalize many existing tree layouts

The Weighted Edge Product correlates with cache miss reduction

Abstract

This paper proposes a general framework for generating cache-oblivious layouts for binary search trees. A cache-oblivious layout attempts to minimize cache misses on any hierarchical memory, independent of the number of memory levels and attributes at each level such as cache size, line size, and replacement policy. Recursively partitioning a tree into contiguous subtrees and prescribing an ordering amongst the subtrees, Hierarchical Layouts generalize many commonly used layouts for trees such as in-order, pre-order and breadth-first. They also generalize the various flavors of the van Emde Boas layout, which have previously been used as cache-oblivious layouts. Hierarchical Layouts thus unify all previous attempts at deriving layouts for search trees. The paper then derives a new locality measure (the Weighted Edge Product) that mimics the probability of cache misses at multiple levels, and shows that layouts that reduce this measure perform better. We analyze the various degrees of freedom in the construction of Hierarchical Layouts, and investigate the relative effect of each of these decisions in the construction of cache-oblivious layouts. Optimizing the Weighted Edge Product for complete binary search trees, we introduce the MinWEP layout, and show that it outperforms previously used cache-oblivious layouts by almost 20%.