Algorithms for Efficient Mining of Statistically Significant Attribute Association Information

TL;DR

This paper presents efficient algorithms for mining statistically significant attribute associations, such as correlation and interaction information, to better understand data structure and attribute interdependencies.

Contribution

It introduces novel data mining methods to efficiently identify non-redundant, significant attribute associations, addressing computational challenges in high-dimensional data.

Findings

Developed algorithms for significant association detection

Reduced redundancy in mined attribute sets

Enhanced interpretability of data models

Abstract

Knowledge of the association information between the attributes in a data set provides insight into the underlying structure of the data and explains the relationships (independence, synergy, redundancy) between the attributes and class (if present). Complex models learnt computationally from the data are more interpretable to a human analyst when such interdependencies are known. In this paper, we focus on mining two types of association information among the attributes - correlation information and interaction information for both supervised (class attribute present) and unsupervised analysis (class attribute absent). Identifying the statistically significant attribute associations is a computationally challenging task - the number of possible associations increases exponentially and many associations contain redundant information when a number of correlated attributes are present. In this paper, we explore efficient data mining methods to discover non-redundant attribute sets that contain significant association information indicating the presence of informative patterns in the data.