Towards a universal representation of statistical dependence

TL;DR

This paper proposes a formal, universal framework for representing statistical dependence between two variables, addressing ambiguities in traditional measures and aligning with intuitive understanding.

Contribution

It introduces a rigorous definition of dependence and a universal representation that unifies various dependence concepts and measures.

Findings

Defines a formal dependence concept consistent with intuition

Develops a universal dependence representation applicable to any bivariate distribution

Shows the proposed dependence measure satisfies Renyi's postulates

Abstract

Dependence is undoubtedly a central concept in statistics. Though, it proves difficult to locate in the literature a formal definition which goes beyond the self-evident 'dependence = non-independence'. This absence has allowed the term 'dependence' and its declination to be used vaguely and indiscriminately for qualifying a variety of disparate notions, leading to numerous incongruities. For example, the classical Pearson's, Spearman's or Kendall's correlations are widely regarded as 'dependence measures' of major interest, in spite of returning 0 in some cases of deterministic relationships between the variables at play, evidently not measuring dependence at all. Arguing that research on such a fundamental topic would benefit from a slightly more rigid framework, this paper suggests a general definition of the dependence between two random variables defined on the same probability space. Natural enough for aligning with intuition, that definition is still sufficiently precise for allowing unequivocal identification of a 'universal' representation of the dependence structure of any bivariate distribution. Links between this representation and familiar concepts are highlighted, and ultimately, the idea of a dependence measure based on that universal representation is explored and shown to satisfy Renyi's postulates.