Assessing Monotone Dependence: Area Under the Curve Meets Rank Correlation

TL;DR

This paper introduces a unified measure of monotone dependence, bridging continuous and dichotomous data, by developing the asymmetric grade correlation and related tests, with applications in weather prediction and language models.

Contribution

It develops a new, unified measure called CMA that generalizes Spearman's Rho and AUC, with theoretical foundations, estimators, and hypothesis tests for all linearly ordered outcomes.

Findings

AGC equals Spearman's Rho for continuous variables

CMA equals AUC when Y is dichotomous

Central limit theorems and DeLong-type tests are established

Abstract

The assessment of monotone dependence between random variables $X$ and $Y$ is a classical problem in statistics and a gamut of application domains. Consequently, researchers have sought measures of association that are invariant under strictly increasing transformations of the margins, with the extant literature being splintered. Rank correlation coefficients, such as Spearman's Rho and Kendall's Tau, have been studied at great length in the statistical literature, mostly under the assumption that $X$ and $Y$ are continuous. In the case of a dichotomous outcome $Y$, receiver operating characteristic analysis and the asymmetric area under the curve (AUC) measure are used to assess monotone dependence of $Y$ on a covariate $X$. Here we unify and extend thus far disconnected strands of literature, by developing common population level theory, estimators, and tests that bridge continuous and dichotomous settings and apply to all linearly ordered outcomes. In particular, we introduce asymmetric grade correlation, AGC$(X,Y)$, as the covariance of the mid distribution function transforms, or grades, of $X$ and $Y$, divided by the variance of the grade of $Y$. The coefficient of monotone association then is CMA$(X,Y) = \frac{1}{2} ($AGC$(X,Y) + 1)$. When $X$ and $Y$ are continuous, AGC is symmetric and equals Spearman's Rho. When $Y$ is dichotomous, CMA equals AUC. We establish central limit theorems for the sample versions of AGC and CMA and develop a test of DeLong type for the equality of AGC or CMA values with a shared outcome $Y$. In case studies, we apply the new measures to assess progress in data-driven weather prediction, and to evaluate methods of uncertainty quantification for large language models.