Copula-based Kernel Dependency Measures

TL;DR

This paper introduces a copula-based dependence measure extending MMD, which is invariant to marginal transformations, robust, and useful for feature selection and distribution embedding.

Contribution

The paper develops a novel copula-based dependence measure extending MMD, with proven properties, robust estimation, and applications in independence testing and feature selection.

Findings

Measure is invariant to monotonic transformations

Estimator is consistent and robust to outliers

Effective in feature selection and distribution embedding

Abstract

The paper presents a new copula based method for measuring dependence between random variables. Our approach extends the Maximum Mean Discrepancy to the copula of the joint distribution. We prove that this approach has several advantageous properties. Similarly to Shannon mutual information, the proposed dependence measure is invariant to any strictly increasing transformation of the marginal variables. This is important in many applications, for example in feature selection. The estimator is consistent, robust to outliers, and uses rank statistics only. We derive upper bounds on the convergence rate and propose independence tests too. We illustrate the theoretical contributions through a series of experiments in feature selection and low-dimensional embedding of distributions.