Clustering and Classification via Cluster-Weighted Factor Analyzers

TL;DR

This paper introduces a family of sixteen cluster-weighted factor analyzers models that improve clustering and classification in high-dimensional data by incorporating latent factor structures and efficient estimation algorithms.

Contribution

It proposes a new family of CWFA models with constraints and an effective EM algorithm, enhancing model-based clustering and classification for high-dimensional data.

Findings

Models demonstrate excellent clustering and classification performance.

Algorithm effectively recovers parameters from artificial and real data.

Hierarchical initialization guarantees proper model ranking.

Abstract

In model-based clustering and classification, the cluster-weighted model constitutes a convenient approach when the random vector of interest constitutes a response variable Y and a set p of explanatory variables X. However, its applicability may be limited when p is high. To overcome this problem, this paper assumes a latent factor structure for X in each mixture component. This leads to the cluster-weighted factor analyzers (CWFA) model. By imposing constraints on the variance of Y and the covariance matrix of X, a novel family of sixteen CWFA models is introduced for model-based clustering and classification. The alternating expectation-conditional maximization algorithm, for maximum likelihood estimation of the parameters of all the models in the family, is described; to initialize the algorithm, a 5-step hierarchical procedure is proposed, which uses the nested structures of the models within the family and thus guarantees the natural ranking among the sixteen likelihoods. Artificial and real data show that these models have very good clustering and classification performance and that the algorithm is able to recover the parameters very well.