Improved Compressed String Dictionaries

TL;DR

This paper presents a new family of compressed data structures that efficiently store and query large string dictionaries, achieving better compression and competitive query times for URL collections and RDF datasets.

Contribution

It introduces a novel combination of hierarchical Front-coding and longest-common-prefix ideas for improved string dictionary compression.

Findings

Achieves better compression than existing methods.

Provides fast query times suitable for large datasets.

Effective in Web graph and RDF data applications.

Abstract

We introduce a new family of compressed data structures to efficiently store and query large string dictionaries in main memory. Our main technique is a combination of hierarchical Front-coding with ideas from longest-common-prefix computation in suffix arrays. Our data structures yield relevant space-time tradeoffs in real-world dictionaries. We focus on two domains where string dictionaries are extensively used and efficient compression is required: URL collections, a key element in Web graphs and applications such as Web mining; and collections of URIs and literals, the basic components of RDF datasets. Our experiments show that our data structures achieve better compression than the state-of-the-art alternatives while providing very competitive query times.