Two-cluster test

TL;DR

This paper introduces a novel two-cluster test to accurately determine if two data subsets originate from the same cluster, addressing inflated error rates of traditional tests in clustering contexts.

Contribution

It presents a new significance testing method based on boundary points, reducing Type-I errors and applicable to interpretability and hierarchical clustering.

Findings

Significantly reduces Type-I error rate compared to classic tests

Effective in synthetic and real data experiments

Applicable in tree-based interpretable clustering

Abstract

Cluster analysis is a fundamental research issue in statistics and machine learning. In many modern clustering methods, we need to determine whether two subsets of samples come from the same cluster. Since these subsets are usually generated by certain clustering procedures, the deployment of classic two-sample tests in this context would yield extremely smaller p-values, leading to inflated Type-I error rate. To overcome this bias, we formally introduce the two-cluster test issue and argue that it is a totally different significance testing issue from conventional two-sample test. Meanwhile, we present a new method based on the boundary points between two subsets to derive an analytical p-value for the purpose of significance quantification. Experiments on both synthetic and real data sets show that the proposed test is able to significantly reduce the Type-I error rate, in comparison with several classic two-sample testing methods. More importantly, the practical usage of such two-cluster test is further verified through its applications in tree-based interpretable clustering and significance-based hierarchical clustering.