Clustering With Pairwise Relationships: A Generative Approach

TL;DR

This paper introduces a probabilistic, generative model for semi-supervised clustering that incorporates user-defined pairwise relationships without heuristic constraints, improving flexibility and optimality.

Contribution

It presents a novel generative framework for constrained clustering that models pairwise relations probabilistically, using EM for parameter estimation and providing closed-form solutions for standard distributions.

Findings

Probabilistic modeling of pairwise constraints improves clustering accuracy.

The approach is flexible for various underlying distributions.

Closed-form solutions facilitate efficient parameter updates.

Abstract

Semi-supervised learning (SSL) has become important in current data analysis applications, where the amount of unlabeled data is growing exponentially and user input remains limited by logistics and expense. Constrained clustering, as a subclass of SSL, makes use of user input in the form of relationships between data points (e.g., pairs of data points belonging to the same class or different classes) and can remarkably improve the performance of unsupervised clustering in order to reflect user-defined knowledge of the relationships between particular data points. Existing algorithms incorporate such user input, heuristically, as either hard constraints or soft penalties, which are separate from any generative or statistical aspect of the clustering model; this results in formulations that are suboptimal and not sufficiently general. In this paper, we propose a principled, generative approach to probabilistically model, without ad hoc penalties, the joint distribution given by user-defined pairwise relations. The proposed model accounts for general underlying distributions without assuming a specific form and relies on expectation-maximization for model fitting. For distributions in a standard form, the proposed approach results in a closed-form solution for updated parameters.