A Study on Variants of Conventional, Fuzzy, and Nullspace-Based Independence Criteria for Improving Supervised and Unsupervised Learning

TL;DR

This paper reviews and proposes new independence criteria to enhance supervised and unsupervised learning, demonstrating improved performance and interpretability over traditional methods in various data settings.

Contribution

It introduces three novel independence criteria and applies them to develop new dimensionality reduction methods that outperform existing baselines.

Findings

Proposed methods outperform baseline techniques like tSNE, PCA, and VAE.

Enhanced interpretability in both linear and nonlinear models.

Methods show improved contrast and accuracy in experiments.

Abstract

Unsupervised and supervised learning methods conventionally use kernels to capture nonlinearities inherent in data structure. However experts have to ensure their proposed nonlinearity maximizes variability and capture inherent diversity of data. We reviewed all independence criteria to design unsupervised learners. Then we proposed 3 independence criteria and used them to design unsupervised and supervised dimensionality reduction methods. We evaluated contrast, accuracy and interpretability of these methods in both linear and neural nonlinear settings. The results show that the methods have outperformed the baseline (tSNE, PCA, regularized LDA, VAE with (un)supervised learner and layer sharing) and opened a new line of interpretable machine learning (ML) for the researchers.