Efficient tree-structured categorical retrieval

TL;DR

This paper introduces efficient data structures and algorithms for fast retrieval of documents within a category tree based on pattern matching, optimizing for space and query time in hierarchical classification systems.

Contribution

It presents novel solutions for categorical document retrieval in tree-structured categories, balancing space efficiency and query performance.

Findings

Achieves $O(|p|+t)$ query time with space proportional to document length and category tree size.

Provides alternative solutions with reduced space at the cost of slightly increased query time.

Offers practical methods for hierarchical document retrieval in taxonomy-based systems.

Abstract

We study a document retrieval problem in the new framework where $D$ text documents are organized in a {\em category tree} with a pre-defined number $h$ of categories. This situation occurs e.g. with taxomonic trees in biology or subject classification systems for scientific literature. Given a string pattern $p$ and a category (level in the category tree), we wish to efficiently retrieve the $t$ \emph{categorical units} containing this pattern and belonging to the category. We propose several efficient solutions for this problem. One of them uses $n(\log\sigma(1+o(1))+\log D+O(h)) + O(\Delta)$ bits of space and $O(|p|+t)$ query time, where $n$ is the total length of the documents, $\sigma$ the size of the alphabet used in the documents and $\Delta$ is the total number of nodes in the category tree. Another solution uses $n(\log\sigma(1+o(1))+O(\log D))+O(\Delta)+O(D\log n)$ bits of space and $O(|p|+t\log D)$ query time. We finally propose other solutions which are more space-efficient at the expense of a slight increase in query time.