Predictive Overlapping Co-Clustering

TL;DR

This paper introduces Predictive Overlapping Co-Clustering (POCC), an optimization-based method that enhances predictive analysis by generating optimal co-clusters with high predictive power, outperforming traditional clustering techniques.

Contribution

The paper presents a novel POCC algorithm that maximizes predictive power of co-clusters, improving data analysis in high-dimensional, heterogeneous datasets.

Findings

POCC outperforms K-means and Spectral co-clustering on real datasets.

POCC achieves higher precision, recall, and F-measure.

Effective in applications like healthcare, recommendation systems, and community detection.

Abstract

In the past few years co-clustering has emerged as an important data mining tool for two way data analysis. Co-clustering is more advantageous over traditional one dimensional clustering in many ways such as, ability to find highly correlated sub-groups of rows and columns. However, one of the overlooked benefits of co-clustering is that, it can be used to extract meaningful knowledge for various other knowledge extraction purposes. For example, building predictive models with high dimensional data and heterogeneous population is a non-trivial task. Co-clusters extracted from such data, which shows similar pattern in both the dimension, can be used for a more accurate predictive model building. Several applications such as finding patient-disease cohorts in health care analysis, finding user-genre groups in recommendation systems and community detection problems can benefit from co-clustering technique that utilizes the predictive power of the data to generate co-clusters for improved data analysis. In this paper, we present the novel idea of Predictive Overlapping Co-Clustering (POCC) as an optimization problem for a more effective and improved predictive analysis. Our algorithm generates optimal co-clusters by maximizing predictive power of the co-clusters subject to the constraints on the number of row and column clusters. In this paper precision, recall and f-measure have been used as evaluation measures of the resulting co-clusters. Results of our algorithm has been compared with two other well-known techniques - K-means and Spectral co-clustering, over four real data set namely, Leukemia, Internet-Ads, Ovarian cancer and MovieLens data set. The results demonstrate the effectiveness and utility of our algorithm POCC in practice.