Optimal Rescaling and the Mahalanobis Distance

TL;DR

This paper generalizes the Mahalanobis distance by introducing a cross-entropy-based measure to find an optimal affine rescaling of data, improving upon classical normalization methods with a new formula for the transformation.

Contribution

It introduces a cross-entropy framework to optimize data rescaling, extending the Mahalanobis approach with a new formula for the covariance matrix that accounts for additional conditions.

Findings

Provides a new formula for optimal affine rescaling based on cross-entropy.

Shows that the classical Mahalanobis transformation is optimal when the origin is at the mean.

Generalizes the Mahalanobis distance to include additional constraints.

Abstract

One of the basic problems in data analysis lies in choosing the optimal rescaling (change of coordinate system) to study properties of a given data-set $Y$. The classical Mahalanobis approach has its basis in the classical normalization/rescaling formula $Y \ni y \to \Sigma_Y^{-1/2} \cdot (y-\mathrm{m}_Y)$, where $\mathrm{m}_Y$ denotes the mean of $Y$ and $\Sigma_Y$ the covariance matrix . Based on the cross-entropy we generalize this approach and define the parameter which measures the fit of a given affine rescaling of $Y$ compared to the Mahalanobis one. This allows in particular to find an optimal change of coordinate system which satisfies some additional conditions. In particular we show that in the case when we put origin of coordinate system in $ \mathrm{m} $ the optimal choice is given by the transformation $Y \ni y \to \Sigma_Y^{-1/2} \cdot (y-\mathrm{m}_Y)$, where $$ \Sigma=\Sigma_Y(\Sigma_Y-\frac{(\mathrm{m}-\mathrm{m}_Y)(\mathrm{m}-\mathrm{m}_Y)^T}{1+\|\mathrm{m}-\mathrm{m}_Y\|_{\Sigma_Y}^2})^{-1}\Sigma_Y. $$