Ranked Document Retrieval in (Almost) No Space

TL;DR

This paper introduces a novel data structure that enables fast ranked document retrieval within minimal additional space, significantly improving query speed while maintaining a highly compressed index.

Contribution

It presents an enhancement to wavelet trees on bytecodes (WTBCs) that supports ranked queries using only 6%-18% of the compressed space, enabling retrieval in tens of milliseconds.

Findings

Supports ranked conjunctive and disjunctive queries

Uses only 6%-18% of compressed space

Achieves query times in tens of milliseconds

Abstract

Ranked document retrieval is a fundamental task in search engines. Such queries are solved with inverted indexes that require additional 45%-80% of the compressed text space, and take tens to hundreds of microseconds per query. In this paper we show how ranked document retrieval queries can be solved within tens of milliseconds using essentially no extra space over an in-memory compressed representation of the document collection. More precisely, we enhance wavelet trees on bytecodes (WTBCs), a data structure that rearranges the bytes of the compressed collection, so that they support ranked conjunctive and disjunctive queries, using just 6%-18% of the compressed text space.