Data Structure Lower Bounds for Document Indexing Problems

TL;DR

This paper establishes strong, unconditional lower bounds for document indexing and pattern matching problems using the pointer machine model, closely matching existing upper bounds and advancing theoretical understanding.

Contribution

It demonstrates the effectiveness of the pointer machine model in proving lower bounds for complex document indexing problems, which were previously difficult to analyze.

Findings

Lower bounds match known space-query time trade-offs

Pointer machine model proves high, unconditional lower bounds

Results apply to various pattern matching and document indexing problems

Abstract

We study data structure problems related to document indexing and pattern matching queries and our main contribution is to show that the pointer machine model of computation can be extremely useful in proving high and unconditional lower bounds that cannot be obtained in any other known model of computation with the current techniques. Often our lower bounds match the known space-query time trade-off curve and in fact for all the problems considered, there is a very good and reasonable match between the our lower bounds and the known upper bounds, at least for some choice of input parameters. The problems that we consider are set intersection queries (both the reporting variant and the semi-group counting variant), indexing a set of documents for two-pattern queries, or forbidden- pattern queries, or queries with wild-cards, and indexing an input set of gapped-patterns (or two-patterns) to find those matching a document given at the query time.