Selective Clustering Annotated using Modes of Projections

TL;DR

SCAMP is a novel non-parametric clustering algorithm that identifies clusters based on shape constraints along projections without pre-specifying the number of clusters, and includes annotation and uncertainty assessment features.

Contribution

SCAMP introduces a shape-constrained search approach for clustering that does not require pre-setting the number of clusters and provides annotation and uncertainty measures.

Findings

Effective in identifying clusters without pre-defining their number.

Provides meaningful annotations describing cluster characteristics.

Available implementation in C++ with R interface.

Abstract

Selective clustering annotated using modes of projections (SCAMP) is a new clustering algorithm for data in $\mathbb{R}^p$. SCAMP is motivated from the point of view of non-parametric mixture modeling. Rather than maximizing a classification likelihood to determine cluster assignments, SCAMP casts clustering as a search and selection problem. One consequence of this problem formulation is that the number of clusters is $\textbf{not}$ a SCAMP tuning parameter. The search phase of SCAMP consists of finding sub-collections of the data matrix, called candidate clusters, that obey shape constraints along each coordinate projection. An extension of the dip test of Hartigan and Hartigan (1985) is developed to assist the search. Selection occurs by scoring each candidate cluster with a preference function that quantifies prior belief about the mixture composition. Clustering proceeds by selecting candidates to maximize their total preference score. SCAMP concludes by annotating each selected cluster with labels that describe how cluster-level statistics compare to certain dataset-level quantities. SCAMP can be run multiple times on a single data matrix. Comparison of annotations obtained across iterations provides a measure of clustering uncertainty. Simulation studies and applications to real data are considered. A C++ implementation with R interface is $\href{https://github.com/RGLab/scamp}{available\ online}$.