Cluster Weighted Model Based on TSNE algorithm for High-Dimensional Data

TL;DR

This paper introduces a TSNE-based dimensionality reduction technique to improve the performance of Cluster Weighted Models (CWMs) in high-dimensional data, demonstrating enhanced accuracy and efficiency on real datasets.

Contribution

It proposes integrating TSNE with CWMs to better handle high-dimensional data, addressing limitations of existing parsimonious techniques.

Findings

TSNE improves CWM performance in high-dimensional spaces

The combined method enhances clustering accuracy on real datasets

The approach reduces computational complexity in high-dimensional clustering

Abstract

Similar to many Machine Learning models, both accuracy and speed of the Cluster weighted models (CWMs) can be hampered by high-dimensional data, leading to previous works on a parsimonious technique to reduce the effect of "Curse of dimensionality" on mixture models. In this work, we review the background study of the cluster weighted models (CWMs). We further show that parsimonious technique is not sufficient for mixture models to thrive in the presence of huge high-dimensional data. We discuss a heuristic for detecting the hidden components by choosing the initial values of location parameters using the default values in the "FlexCWM" R package. We introduce a dimensionality reduction technique called T-distributed stochastic neighbor embedding (TSNE) to enhance the parsimonious CWMs in high-dimensional space. Originally, CWMs are suited for regression but for classification purposes, all multi-class variables are transformed logarithmically with some noise. The parameters of the model are obtained via expectation maximization algorithm. The effectiveness of the discussed technique is demonstrated using real data sets from different fields.