Adjust Pearson's $r$ to Measure Arbitrary Monotone Dependence

TL;DR

This paper introduces a new correlation measure called rearrangement correlation, which accurately captures both linear and nonlinear monotone dependencies, improving upon Pearson's r and other existing measures through theoretical refinement and empirical validation.

Contribution

The authors derive a tighter inequality than Cauchy-Schwarz, propose the rearrangement correlation, and demonstrate its effectiveness in measuring nonlinear monotone dependence.

Findings

Rearrangement correlation outperforms classical measures in nonlinear scenarios.

Tighter bounds expand the capture range of dependence measures.

Simulation and real-life tests confirm improved accuracy of the new measure.

Abstract

Pearson's r, the most widely-used correlation coefficient, is traditionally regarded as exclusively capturing linear dependence, leading to its discouragement in contexts involving nonlinear relationships. However, recent research challenges this notion, suggesting that Pearson's r should not be ruled out a priori for measuring nonlinear monotone relationships. Pearson's r is essentially a scaled covariance, rooted in the renowned Cauchy-Schwarz Inequality. Our findings reveal that different scaling bounds yield coefficients with different capture ranges, and interestingly, tighter bounds actually expand these ranges. We derive a tighter inequality than Cauchy-Schwarz Inequality, leverage it to refine Pearson's r, and propose a new correlation coefficient, i.e., rearrangement correlation. This coefficient is able to capture arbitrary monotone relationships, both linear and nonlinear ones. It reverts to Pearson's r in linear scenarios. Simulation experiments and real-life investigations show that the rearrangement correlation is more accurate in measuring nonlinear monotone dependence than the three classical correlation coefficients, and other recently proposed dependence measures.