Clustering Via Finite Nonparametric ICA Mixture Models

TL;DR

This paper introduces a novel mixture model that integrates ICA structure with non-parametric clustering, optimizing it with a new algorithm and demonstrating its effectiveness in unsupervised learning and image processing.

Contribution

It extends finite mixture models by removing the conditional independence assumption and incorporating ICA, along with a new optimization algorithm and R package implementation.

Findings

Effective clustering in unsupervised learning tasks

Improved image processing capabilities

Successful application of the NSMM-ICA algorithm

Abstract

We propose an extension of non-parametric multivariate finite mixture models by dropping the standard conditional independence assumption and incorporating the independent component analysis (ICA) structure instead. We formulate an objective function in terms of penalized smoothed Kullback Leibler distance and introduce the nonlinear smoothed majorization-minimization independent component analysis (NSMM-ICA) algorithm for optimizing this function and estimating the model parameters. We have implemented a practical version of this algorithm, which utilizes the FastICA algorithm, in the R package icamix. We illustrate this new methodology using several applications in unsupervised learning and image processing.