Optimal Clustering with Missing Values

TL;DR

This paper introduces a novel optimal clustering framework that directly handles missing values in data, especially in biomedical contexts, by integrating the missing value mechanism into the clustering process, avoiding imputation.

Contribution

The paper develops a new optimal clustering method that incorporates missing value mechanisms into the model, demonstrated on Gaussian data and RNA-seq data, outperforming traditional imputation-based methods.

Findings

Outperforms traditional methods on synthetic data

Achieves lower clustering errors on RNA-seq data

Eliminates the need for data imputation in clustering

Abstract

Missing values frequently arise in modern biomedical studies due to various reasons, including missing tests or complex profiling technologies for different omics measurements. Missing values can complicate the application of clustering algorithms, whose goals are to group points based on some similarity criterion. A common practice for dealing with missing values in the context of clustering is to first impute the missing values, and then apply the clustering algorithm on the completed data. We consider missing values in the context of optimal clustering, which finds an optimal clustering operator with reference to an underlying random labeled point process (RLPP). We show how the missing-value problem fits neatly into the overall framework of optimal clustering by incorporating the missing value mechanism into the random labeled point process and then marginalizing out the missing-value process. In particular, we demonstrate the proposed framework for the Gaussian model with arbitrary covariance structures. Comprehensive experimental studies on both synthetic and real-world RNA-seq data show the superior performance of the proposed optimal clustering with missing values when compared to various clustering approaches. Optimal clustering with missing values obviates the need for imputation-based pre-processing of the data, while at the same time possessing smaller clustering errors.