On the Monotonicity of the Copula Entropy

TL;DR

This paper establishes a monotonic relationship between mutual information and copula dependence parameters, providing a theoretical insight and practical tools for scalable model selection in probabilistic modeling.

Contribution

It introduces a novel theoretical relationship linking mutual information and copula parameters, enabling efficient model selection methods.

Findings

Monotonic relationship between mutual information and copula parameters.

Efficient proxy for expected likelihood in model selection.

Applicable to a wide range of copula families.

Abstract

Understanding the way in which random entities interact is of key interest in numerous scientific fields. This can range from a full characterization of the joint distribution to single scalar summary statistics. In this work we identify a novel relationship between the ubiquitous Shannon's mutual information measure and the central tool for capturing real-valued non-Gaussian distributions, namely the framework of copulas. Specifically, we establish a monotonic relationship between the mutual information and the copula dependence parameter, for a wide range of copula families. In addition to the theoretical novelty, our result gives rise to highly efficient proxy to the expected likelihood, which in turn allows for scalable model selection (e.g. when learning probabilistic graphical models).