Weighted Spectral Cluster Ensemble

TL;DR

This paper introduces Weighted Spectral Cluster Ensemble (WSCE), a robust clustering framework that leverages community detection concepts and graph-based methods to improve ensemble performance without relying on heuristic thresholds.

Contribution

It proposes a novel WSCE framework using modularity for diversity estimation and a new spectral clustering method, addressing key issues in existing cluster ensemble techniques.

Findings

WSCE outperforms state-of-the-art methods in various datasets.

The approach effectively estimates diversity without thresholding.

Experimental results validate the robustness of the proposed method.

Abstract

Clustering explores meaningful patterns in the non-labeled data sets. Cluster Ensemble Selection (CES) is a new approach, which can combine individual clustering results for increasing the performance of the final results. Although CES can achieve better final results in comparison with individual clustering algorithms and cluster ensemble methods, its performance can be dramatically affected by its consensus diversity metric and thresholding procedure. There are two problems in CES: 1) most of the diversity metrics is based on heuristic Shannon's entropy and 2) estimating threshold values are really hard in practice. The main goal of this paper is proposing a robust approach for solving the above mentioned problems. Accordingly, this paper develops a novel framework for clustering problems, which is called Weighted Spectral Cluster Ensemble (WSCE), by exploiting some concepts from community detection arena and graph based clustering. Under this framework, a new version of spectral clustering, which is called Two Kernels Spectral Clustering, is used for generating graphs based individual clustering results. Further, by using modularity, which is a famous metric in the community detection, on the transformed graph representation of individual clustering results, our approach provides an effective diversity estimation for individual clustering results. Moreover, this paper introduces a new approach for combining the evaluated individual clustering results without the procedure of thresholding. Experimental study on varied data sets demonstrates that the prosed approach achieves superior performance to state-of-the-art methods.