Stagewise Learning for Sparse Clustering of Discretely-Valued Data

TL;DR

This paper introduces a stagewise EM algorithm driven by expert data, which iteratively develops mixture components based on mutual information, improving clustering performance in noisy, crowdsourcing, and neuroscience data scenarios.

Contribution

The paper proposes a novel stagewise EM algorithm that incrementally builds mixture components using mutual information, enhancing clustering accuracy in noisy data environments.

Findings

Stagewise EM outperforms traditional initialization methods.

It achieves results comparable to knowing the true distribution.

Effective in crowdsourcing and neuroscience applications.

Abstract

The performance of EM in learning mixtures of product distributions often depends on the initialization. This can be problematic in crowdsourcing and other applications, e.g. when a small number of 'experts' are diluted by a large number of noisy, unreliable participants. We develop a new EM algorithm that is driven by these experts. In a manner that differs from other approaches, we start from a single mixture class. The algorithm then develops the set of 'experts' in a stagewise fashion based on a mutual information criterion. At each stage EM operates on this subset of the players, effectively regularizing the E rather than the M step. Experiments show that stagewise EM outperforms other initialization techniques for crowdsourcing and neurosciences applications, and can guide a full EM to results comparable to those obtained knowing the exact distribution.