A family of Chatterjee's correlation coefficients and their properties

TL;DR

This paper introduces a flexible family of correlation coefficients based on Chatterjee's measure, capable of capturing complex dependencies between variables beyond linear or monotone relationships, with proven convergence and desirable properties.

Contribution

It proposes a new family of correlation coefficients parameterized by functions, generalizing Chatterjee's coefficient, with theoretical convergence and property guarantees.

Findings

The family converges almost surely to a deterministic limit.

The limit satisfies properties of range, independence, and functional dependence detection.

Numerical experiments demonstrate practical usefulness and flexibility.

Abstract

Quantifying the strength of functional dependence between random scalars $X$ and $Y$ is an important statistical problem. While many existing correlation coefficients excel in identifying linear or monotone functional dependence, they fall short in capturing general non-monotone functional relationships. In response, we propose a family of correlation coefficients $\xi^{(h,F)}_n$, characterized by a continuous bivariate function $h$ and a cdf function $F$. By offering a range of selections for $h$ and $F$, $\xi^{(h,F)}_n$ encompasses a diverse class of novel correlation coefficients, while also incorporates the Chatterjee's correlation coefficient (Chatterjee, 2021) as a special case. We prove that $\xi^{(h,F)}_n$ converges almost surely to a deterministic limit $\xi^{(h,F)}$ as sample size $n$ approaches infinity. In addition, under appropriate conditions imposed on $h$ and $F$, the limit $\xi^{(h,F)}$ satisfies the three appealing properties: (P1). it belongs to the range of $[0,1]$; (P2). it equals 1 if and only if $Y$ is a measurable function of $X$; and (P3). it equals 0 if and only if $Y$ is independent of $X$. As amplified by our numerical experiments, our proposals provide practitioners with a variety of options to choose the most suitable correlation coefficient tailored to their specific practical needs.