Multi-Dimensional Data Compression and Query Processing in Array Databases

TL;DR

This paper introduces SEACOW, a specialized compression scheme for multidimensional array data that enhances storage efficiency and query performance by considering data access patterns and embedding a synopsis for indexing.

Contribution

The paper presents SEACOW, a novel array compression scheme tailored for array databases, improving both compression rates and query efficiency over existing methods.

Findings

SEACOW achieves higher compression rates than existing schemes.

The embedded synopsis significantly improves analytical query performance.

Experiments on scientific datasets validate the effectiveness of SEACOW.

Abstract

In recent times, the production of multidimensional data in various domains and their storage in array databases has witnessed a sharp increase; this rapid growth in data volumes necessitates compression in array databases. However, existing compression schemes used in array databases are general-purpose and not designed specifically for the databases. They could degrade query performance with complex analytical tasks, which incur huge computing costs. Thus, a compression scheme that considers the workflow of array databases is required. This study presents a compression scheme, SEACOW, for storing and querying multidimensional array data. The scheme is specially designed to be efficient for both dimension-based and value-based exploration. It considers data access patterns for exploration queries and embeds a synopsis, which can be utilized as an index, in the compressed array. In addition, we implement an array storage system, namely MSDB, to perform experiments. We evaluate query performance on real scientific datasets and compared it with those of existing compression schemes. Finally, our experiments demonstrate that SEACOW provides high compression rates compared to existing compression schemes, and the synopsis improves analytical query processing performance.