Semiparametric estimation of mutual information and related criteria : optimal test of independence

TL;DR

This paper develops semiparametric independence tests based on phi-mutual information measures, establishing their theoretical properties and demonstrating the optimality of the Kullback-Leibler mutual information test through asymptotic efficiency and simulations.

Contribution

It introduces a novel semiparametric framework for estimating mutual information and testing independence, with proven optimality and applications to model selection in dependency models.

Findings

The proposed tests are consistent and asymptotically efficient.

The semiparametric Kullback-Leibler mutual information test is shown to be optimal.

Numerical simulations confirm the theoretical advantages of the proposed methods.

Abstract

We derive independence tests by means of dependence measures thresholding in a semiparametric context. Precisely, estimates of phi-mutual informations, associated to phi-divergences between a joint distribution and the product distribution of its margins, are derived through the dual representation of phi-divergences. The asymptotic properties of the proposed estimates are established, including consistency, asymptotic distributions and large deviations principle. The obtained tests of independence are compared via their relative asymptotic Bahadur efficiency and numerical simulations. It follows that the proposed semiparametric Kullback-Leibler Mutual information test is the optimal one. On the other hand, the proposed approach provides a new method for estimating the Kullback-Leibler mutual information in a semiparametric setting, as well as a model selection procedure in large class of dependency models including semiparametric copulas.