Data-Driven Estimation Of Mutual Information Between Dependent Data

TL;DR

This paper introduces a novel, non-parametric, frequency-domain method for estimating mutual information between dependent data, enabling detection of dependencies and cross-frequency coupling in various models.

Contribution

The paper presents a new frequency-domain mutual information estimator that is data-driven and non-parametric, with a novel metric for dependence detection across frequencies.

Findings

Effective on linear and nonlinear models

Can infer cross-frequency coupling

Provides a tractable frequency-based dependence measure

Abstract

We consider the problem of estimating mutual information between dependent data, an important problem in many science and engineering applications. We propose a data-driven, non-parametric estimator of mutual information in this paper. The main novelty of our solution lies in transforming the data to frequency domain to make the problem tractable. We define a novel metric--mutual information in frequency--to detect and quantify the dependence between two random processes across frequency using Cram\'{e}r's spectral representation. Our solution calculates mutual information as a function of frequency to estimate the mutual information between the dependent data over time. We validate its performance on linear and nonlinear models. In addition, mutual information in frequency estimated as a part of our solution can also be used to infer cross-frequency coupling in the data.