On the Monotonicity of the Copula Entropy
Yaniv Tenzer, Gal Elidan

TL;DR
This paper establishes a monotonic relationship between mutual information and copula dependence parameters, providing a theoretical insight and practical tools for scalable model selection in probabilistic modeling.
Contribution
It introduces a novel theoretical relationship linking mutual information and copula parameters, enabling efficient model selection methods.
Findings
Monotonic relationship between mutual information and copula parameters.
Efficient proxy for expected likelihood in model selection.
Applicable to a wide range of copula families.
Abstract
Understanding the way in which random entities interact is of key interest in numerous scientific fields. This can range from a full characterization of the joint distribution to single scalar summary statistics. In this work we identify a novel relationship between the ubiquitous Shannon's mutual information measure and the central tool for capturing real-valued non-Gaussian distributions, namely the framework of copulas. Specifically, we establish a monotonic relationship between the mutual information and the copula dependence parameter, for a wide range of copula families. In addition to the theoretical novelty, our result gives rise to highly efficient proxy to the expected likelihood, which in turn allows for scalable model selection (e.g. when learning probabilistic graphical models).
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsBayesian Modeling and Causal Inference · Bayesian Methods and Mixture Models · Statistical Methods and Inference
