Statistically Significant Pattern Mining with Ordinal Utility

TL;DR

This paper introduces a novel approach to statistically significant pattern mining that incorporates utility preferences, enabling the discovery of more useful patterns while controlling error rates.

Contribution

It proposes an iterative testing method that integrates utility preferences into SSPM, improving pattern relevance and statistical control over existing methods.

Findings

Discoveries are more useful and numerous in real-world data

The method controls the familywise error rate effectively

It outperforms the standard Tarone-Bonferroni method in experiments

Abstract

Statistically significant patterns mining (SSPM) is an essential and challenging data mining task in the field of knowledge discovery in databases (KDD), in which each pattern is evaluated via a hypothesis test. Our study aims to introduce a preference relation into patterns and to discover the most preferred patterns under the constraint of statistical significance, which has never been considered in existing SSPM problems. We propose an iterative multiple testing procedure that can alternately reject a hypothesis and safely ignore the hypotheses that are less useful than the rejected hypothesis. One advantage of filtering out patterns with low utility is that it avoids consumption of the significance budget by rejection of useless (that is, uninteresting) patterns. This allows the significance budget to be focused on useful patterns, leading to more useful discoveries. We show that the proposed method can control the familywise error rate (FWER) under certain assumptions, that can be satisfied by a realistic problem class in SSPM.\@We also show that the proposed method always discovers a set of patterns that is at least equally or more useful than those discovered using the standard Tarone-Bonferroni method SSPM.\@Finally, we conducted several experiments with both synthetic and real-world data to evaluate the performance of our method. As a result, in the experiments with real-world datasets, the proposed method discovered a larger number of more useful patterns than the existing method for all five conducted tasks.