On Quantifying Dependence: A Framework for Developing Interpretable Measures

TL;DR

This paper introduces a flexible framework for developing and selecting dependence measures focused on interpretability and tailored to specific needs, moving beyond mere detection to quantification of relationships.

Contribution

It proposes a general, adaptable framework for creating dependence measures emphasizing interpretability, integrating information-theoretic tools, and demonstrating applications with real data examples.

Findings

Many existing measures fit within the framework.

The methodology guides the development of tailored dependence measures.

Applications include income-education data and geomagnetic storm analysis.

Abstract

We present a framework for selecting and developing measures of dependence when the goal is the quantification of a relationship between two variables, not simply the establishment of its existence. Much of the literature on dependence measures is focused, at least implicitly, on detection or revolves around the inclusion/exclusion of particular axioms and discussing which measures satisfy said axioms. In contrast, we start with only a few nonrestrictive guidelines focused on existence, range and interpretability, which provide a very open and flexible framework. For quantification, the most crucial is the notion of interpretability, whose foundation can be found in the work of Goodman and Kruskal [Measures of Association for Cross Classifications (1979) Springer], and whose importance can be seen in the popularity of tools such as the $R^2$ in linear regression. While Goodman and Kruskal focused on probabilistic interpretations for their measures, we demonstrate how more general measures of information can be used to achieve the same goal. To that end, we present a strategy for building dependence measures that is designed to allow practitioners to tailor measures to their needs. We demonstrate how many well-known measures fit in with our framework and conclude the paper by presenting two real data examples. Our first example explores U.S. income and education where we demonstrate how this methodology can help guide the selection and development of a dependence measure. Our second example examines measures of dependence for functional data, and illustrates them using data on geomagnetic storms.