Efficiently Enumerating Hitting Sets of Hypergraphs Arising in Data Profiling

TL;DR

This paper presents a new polynomial-delay and polynomial-space algorithm for enumerating minimal hitting sets in hypergraphs, with applications to data profiling, improving upon previous methods and providing theoretical and empirical insights.

Contribution

It introduces a hypergraph enumeration algorithm with delay and space bounds independent of the solution size, and analyzes the complexity of the extension problem for minimal hitting sets.

Findings

Algorithm achieves polynomial delay and space for hypergraph enumeration.

Extension problem for minimal hitting sets is W[3]-complete and nearly optimally solvable.

Empirical results show improved performance on real-world data profiling tasks.

Abstract

The transversal hypergraph problem is the task of enumerating the minimal hitting sets of a hypergraph. It is a long-standing open question whether this can be done in output-polynomial time. For hypergraphs whose solutions have bounded size, Eiter and Gottlob [SICOMP 1995] gave an algorithm that runs in output-polynomial time, but whose space requirement also scales with the output size. We improve this to polynomial delay and polynomial space. More generally, we present an algorithm that on $n$-vertex, $m$-edge hypergraphs has delay $O(m^{k^*+1} n^2)$ and uses $O(mn)$ space, where $k^*$ is the maximum size of any minimal hitting set. Our algorithm is oblivious to $k^*$, a quantity that is hard to compute or even approximate. Central to our approach is the extension problem for minimal hitting sets, deciding for a set $X$ of vertices whether it is contained in any solution. With $|X|$ as parameter, we show that this is one of the first natural problems to be complete for the complexity class $W[3]$. We give an algorithm for the extension problem running in time $O(m^{|X|+1} n)$. We also prove a conditional lower bound under the Strong Exponential Time Hypothesis, showing that this is close to optimal. We apply our enumeration method to the discovery problem of minimal unique column combinations from data profiling. Our empirical evaluation suggests that the algorithm outperforms its worst-case guarantees on hypergraphs stemming from real-world databases.