Wisdom of Crowds cluster ensemble

TL;DR

This paper introduces WOCCE, a novel cluster ensemble framework inspired by the wisdom of crowds, which evaluates and leverages diversity, independence, and decentralization to improve clustering accuracy.

Contribution

The paper proposes a new feedback framework for cluster ensembles that analyzes conditions for collective wisdom, including novel diversity measures and evaluation procedures.

Findings

WOCCE outperforms traditional algorithms in clustering accuracy

The framework effectively assesses diversity and independence in ensemble members

Experimental results demonstrate improved collective decision-making

Abstract

The Wisdom of Crowds is a phenomenon described in social science that suggests four criteria applicable to groups of people. It is claimed that, if these criteria are satisfied, then the aggregate decisions made by a group will often be better than those of its individual members. Inspired by this concept, we present a novel feedback framework for the cluster ensemble problem, which we call Wisdom of Crowds Cluster Ensemble (WOCCE). Although many conventional cluster ensemble methods focusing on diversity have recently been proposed, WOCCE analyzes the conditions necessary for a crowd to exhibit this collective wisdom. These include decentralization criteria for generating primary results, independence criteria for the base algorithms, and diversity criteria for the ensemble members. We suggest appropriate procedures for evaluating these measures, and propose a new measure to assess the diversity. We evaluate the performance of WOCCE against some other traditional base algorithms as well as state-of-the-art ensemble methods. The results demonstrate the efficiency of WOCCE's aggregate decision-making compared to other algorithms.