Supervised Dimensionality Reduction via Distance Correlation Maximization

TL;DR

This paper introduces a new supervised dimensionality reduction method that maximizes distance correlation between features, responses, and covariates without distributional assumptions, demonstrating superior empirical performance.

Contribution

It proposes a novel formulation based on distance correlation for supervised reduction and an optimization algorithm, advancing the state-of-the-art in the field.

Findings

Outperforms existing methods on multiple datasets

Effective without distributional assumptions

Validates the approach's superiority empirically

Abstract

In our work, we propose a novel formulation for supervised dimensionality reduction based on a nonlinear dependency criterion called Statistical Distance Correlation, Szekely et. al. (2007). We propose an objective which is free of distributional assumptions on regression variables and regression model assumptions. Our proposed formulation is based on learning a low-dimensional feature representation $\mathbf{z}$, which maximizes the squared sum of Distance Correlations between low dimensional features $\mathbf{z}$ and response $y$, and also between features $\mathbf{z}$ and covariates $\mathbf{x}$. We propose a novel algorithm to optimize our proposed objective using the Generalized Minimization Maximizaiton method of \Parizi et. al. (2015). We show superior empirical results on multiple datasets proving the effectiveness of our proposed approach over several relevant state-of-the-art supervised dimensionality reduction methods.