Completing and studentising Spearman's correlation in the presence of ties

TL;DR

This paper introduces a new Spearman correlation variant, Kemeny , that maintains desirable statistical properties with ties and discrete data, enabling Student-t null distribution construction without Gaussian assumptions.

Contribution

It proposes a novel Spearman correlation estimator, Kemeny , that satisfies Gauss-Markov conditions and allows Student-t null distribution construction for tied and discrete data.

Findings

Kemeny  satisfies Gauss-Markov conditions with ties and discrete data.

Null hypothesis distribution is Student-t, applicable without Gaussian assumptions.

Simulations show high kurtotic data coverage aligns with theoretical expectations.

Abstract

Non-parametric correlation coefficients have been widely used for analysing arbitrary random variables upon common populations, when requiring an explicit error distribution to be known is an unacceptable assumption. We examine an \(\ell_{2}\) representation of a correlation coefficient (Emond and Mason, 2002) from the perspective of a statistical estimator upon random variables, and verify a number of interesting and highly desirable mathematical properties, mathematically similar to the Whitney embedding of a Hilbert space into the \(\ell_{2}\)-norm space. In particular, we show here that, in comparison to the traditional Spearman (1904) \(\rho\), the proposed Kemeny \(\rho_{\kappa}\) correlation coefficient satisfies Gauss-Markov conditions in the presence or absence of ties, thereby allowing both discrete and continuous marginal random variables. We also prove under standard regularity conditions a number of desirable scenarios, including the construction of a null hypothesis distribution which is Student-t distributed, parallel to standard practice with Pearson's r, but without requiring either continuous random variables nor particular Gaussian errors. Simulations in particular focus upon highly kurtotic data, with highly nominal empirical coverage consistent with theoretical expectation.