Clustering inference in multiple groups

TL;DR

This paper introduces a non-parametric, U-statistics based clustering method for high-dimensional data that effectively identifies three-group structures and assesses their significance, demonstrating superior power and versatility across various applications.

Contribution

The paper develops a novel U-statistics based approach for clustering into three groups with significance testing, extending existing methods and enhancing power in high-dimensional data analysis.

Findings

More statistical power than competing methods in simulations.

Effective in high-dimensional, real-world datasets.

Versatile across different scientific applications.

Abstract

Inference in clustering is paramount to uncovering inherent group structure in data. Clustering methods which assess statistical significance have recently drawn attention owing to their importance for the identification of patterns in high dimensional data with applications in many scientific fields. We present here a U-statistics based approach, specially tailored for high-dimensional data, that clusters the data into three groups while assessing the significance of such partitions. Because our approach stands on the U-statistics based clustering framework of the methods in R package uclust, it inherits its characteristics being a non-parametric method relying on very few assumptions about the data, and thus can be applied to a wide range of dataset. Furthermore our method aims to be a more powerful tool to find the best partitions of the data into three groups when that particular structure is present. In order to do so, we first propose an extension of the test U-statistic and develop its asymptotic theory. Additionally we propose a ternary non-nested significance clustering method. Our approach is tested through multiple simulations and found to have more statistical power than competing alternatives in all scenarios considered. Applications to peripheral blood mononuclear cells and to image recognition shows the versatility of our proposal, presenting a superior performance when compared with other approaches.