Hierarchical Bitmap Indexing for Range and Membership Queries on Multidimensional Arrays

TL;DR

This paper introduces a hierarchical bitmap indexing method for multidimensional arrays that improves query efficiency and reduces memory usage by using n-dimensional sparse trees and adaptive binning, outperforming traditional bitmap indices.

Contribution

The paper presents a novel hierarchical indexing approach using sparse trees and adaptive binning for multidimensional arrays, enhancing query performance and memory efficiency.

Findings

Outperforms conventional bitmap indexing in query speed

Reduces memory requirements through adaptive binning

Efficiently handles range and membership queries

Abstract

Traditional indexing techniques commonly employed in da\-ta\-ba\-se systems perform poorly on multidimensional array scientific data. Bitmap indices are widely used in commercial databases for processing complex queries, due to their effective use of bit-wise operations and space-efficiency. However, bitmap indices apply natively to relational or linearized datasets, which is especially notable in binned or compressed indices. We propose a new method for multidimensional array indexing that overcomes the dimensionality-induced inefficiencies. The hierarchical indexing method is based on $n$-di\-men\-sional sparse trees for dimension partitioning, with bound number of individual, adaptively binned indices for attribute partitioning. This indexing performs well on range involving both dimensions and attributes, as it prunes the search space early, avoids reading entire index data, and does at most a single index traversal. Moreover, the indexing is easily extensible to membership queries. The indexing method was implemented on top of a state of the art bitmap indexing library Fastbit. We show that the hierarchical bitmap index outperforms conventional bitmap indexing built on auxiliary attribute for each dimension. Furthermore, the adaptive binning significantly reduces the amount of bins and therefore memory requirements.