Clustering Data with Nonignorable Missingness using Semi-Parametric Mixture Models

TL;DR

This paper introduces a semi-parametric mixture model for clustering continuous data with non-ignorable missingness, avoiding assumptions on the missingness mechanism and demonstrating effective estimation and identifiability.

Contribution

The paper proposes a novel semi-parametric mixture model for clustering with non-ignorable missing data, including an estimation algorithm and extension to mixed-type data.

Findings

Effective clustering with non-ignorable missingness demonstrated

Algorithm shows monotonic convergence and identifiability under mild conditions

Extension to mixed data types validated on real data

Abstract

We are concerned in clustering continuous data sets subject to non-ignorable missingness. We perform clustering with a specific semi-parametric mixture, under the assumption of conditional independence given the component. The mixture model isused for clustering and not for estimating the density of the full variables (observed and unobserved), thus we do not need other assumptions on the component distribution neither to specify the missingness mechanism. Estimation is performed by maximizing an extension of smoothed likelihood allowing missingness. This optimization is achieved by a Majorization-Minorization algorithm. We illustrate the relevance of our approach by numerical experiments. Under mild assumptions, we show the identifiability of the model defining the distribution of the observed data and the monotony of the algorithm. We also propose an extension of this new method to the case of mixed-type data that we illustrate on a real data set.