Distance-based Chatterjee correlation: a new generalized robust measure of directed association for multivariate real and complex-valued data

TL;DR

This paper introduces a new distance-based Chatterjee correlation measure for multivariate real and complex data, enabling robust, non-parametric, asymmetric assessment of directed associations and causal inference.

Contribution

It extends the original Chatterjee correlation to multivariate and complex data, providing a robust, non-parametric measure of directed association with causal inference capabilities.

Findings

The new measure is robust to outliers.

It can be applied to multivariate real and complex data.

It enables inference of causal direction.

Abstract

Building upon the Chatterjee correlation (2021: J. Am. Stat. Assoc. 116, p2009) for two real-valued variables, this study introduces a generalized measure of directed association between two vector variables, real or complex-valued, and of possibly different dimensions. The new measure is denoted as the "distance-based Chatterjee correlation", owing to the use here of the "distance transformed data" defined in Szekely et al (2007: Ann. Statist. 35, p2769) for the distance correlation. A main property of the new measure, inherited from the original Chatterjee correlation, is its predictive and asymmetric nature: it measures how well one variable can be predicted by the other, asymmetrically. This allows for inferring the causal direction of the association, by using the method of Blobaum et al (2019: PeerJ Comput. Sci. 1, e169). Since the original Chatterjee correlation is based on ranks, it is not available for complex variables, nor for general multivariate data. The novelty of our work is the extension to multivariate real and complex-valued pairs of vectors, offering a robust measure of directed association in a completely non-parametric setting. Informally, the intuitive assumption used here is that distance correlation is mathematically equivalent to Pearson's correlation when applied to "distance transformed" data. The next logical step is to compute Chatterjee's correlation on the same "distance transformed" data, thereby extending the analysis to multivariate vectors of real and complex valued data. As a bonus, the new measure here is robust to outliers, which is not true for the distance correlation of Szekely et al. Additionally, this approach allows for inference regarding the causal direction of the association between the variables.