Making problems tractable on big data via preprocessing with polylog-size output

TL;DR

This paper introduces a new tractability framework, $oxplus'$-tractability, for big data queries with polylog-size preprocessing outputs, refining previous theories to better align with practical data processing constraints.

Contribution

It proposes a novel $oxplus'$-tractability concept restricting preprocessing to produce polylog-size outputs, and analyzes its theoretical properties and relationship to existing complexity classes.

Findings

All PTIME Boolean queries can be made $oxplus'$-tractable.

The set of $oxplus'$-tractable queries is strictly smaller than $oxplus$-tractable queries.

$oxplus'$-tractability defines a new complexity class within PTIME.

Abstract

To provide a dichotomy between those queries that can be made feasible on big data after appropriate preprocessing and those for which preprocessing does not help, Fan et al. developed the $\sqcap$-tractability theory. This theory provides a formal foundation for understanding the tractability of query classes in the context of big data. Along this line, we introduce a novel notion of $\sqcap'$-tractability in this paper. Inspired by some technologies used to deal big data, we place a restriction on preprocessing function, which limits the function to produce a relatively small database as output, at most polylog-size of the input database. At the same time, we bound the redundancy information when re-factorizing data and queries for preprocessing. These changes aim to make our theory more closely linked to practice. We set two complexity classes to denote the classes of Boolean queries that are $\sqcap'$-tractable themselves and that can be made $\sqcap'$-tractable, respectively. Based on a new factorization in our complexity classes, we investigate two reductions, which differ from whether allowing re-factorizing data and query parts. We verify the transitive and compatible properties of the reductions and analysis the complete problems and sizes of the complexity classes. We conclude that all PTIME classes of Boolean queries can be made $\sqcap'$-tractable, similar to that of the $\sqcap$-tractability theory. With a little surprise, we prove that the set of all $\sqcap'$-tractable queries is strictly smaller than that of all $\sqcap$-tractable queries, and thus the set of $\sqcap'$-tractable queries is properly contained in that of PTIME queries. In this way, we attain a new complexity class inside the complexity class of PTIME queries.