Measuring Statistical Dependencies via Maximum Norm and Characteristic Functions

TL;DR

This paper introduces a new statistical dependence measure based on the maximum-norm of characteristic functions difference, capable of detecting complex dependencies in high-dimensional data and integrating into machine learning workflows.

Contribution

The paper proposes a novel dependence measure using maximum-norm of characteristic functions difference, effective in high-dimensional, non-linear data, and applicable in modern machine learning pipelines.

Findings

Effective in high-dimensional, non-linear data

Less affected by curse of dimensionality

Improves supervised feature extraction and neural network regularization

Abstract

In this paper, we focus on the problem of statistical dependence estimation using characteristic functions. We propose a statistical dependence measure, based on the maximum-norm of the difference between joint and product-marginal characteristic functions. The proposed measure can detect arbitrary statistical dependence between two random vectors of possibly different dimensions, is differentiable, and easily integrable into modern machine learning and deep learning pipelines. We also conduct experiments both with simulated and real data. Our simulations show, that the proposed method can measure statistical dependencies in high-dimensional, non-linear data, and is less affected by the curse of dimensionality, compared to the previous work in this line of research. The experiments with real data demonstrate the potential applicability of our statistical measure for two different empirical inference scenarios, showing statistically significant improvement in the performance characteristics when applied for supervised feature extraction and deep neural network regularization. In addition, we provide a link to the accompanying open-source repository https://bit.ly/3d4ch5I.