Delta-Closure Structure for Studying Data Distribution

TL;DR

This paper introduces the $\Delta$-closure structure, a novel framework for analyzing data distribution in binary datasets through generalized closure operators, revealing stable attribute correlations.

Contribution

It proposes $\Delta$-closedness and the $\Delta$-closure structure, providing new tools to characterize and interpret data distributions and attribute correlations.

Findings

The $\Delta$-closure structure is stable across different datasets and sampling methods.

High $\Delta$-classes indicate strong attribute correlations supported by many observations.

The framework generalizes traditional closure operators to better understand data distribution.

Abstract

In this paper, we revisit pattern mining and study the distribution underlying a binary dataset thanks to the closure structure which is based on passkeys, i.e., minimum generators in equivalence classes robust to noise. We introduce $\Delta$-closedness, a generalization of the closure operator, where $\Delta$ measures how a closed set differs from its upper neighbors in the partial order induced by closure. A $\Delta$-class of equivalence includes minimum and maximum elements and allows us to characterize the distribution underlying the data. Moreover, the set of $\Delta$-classes of equivalence can be partitioned into the so-called $\Delta$-closure structure. In particular, a $\Delta$-class of equivalence with a high level demonstrates correlations among many attributes, which are supported by more observations when $\Delta$ is large. In the experiments, we study the $\Delta$-closure structure of several real-world datasets and show that this structure is very stable for large $\Delta$ and does not substantially depend on the data sampling used for the analysis.