Evaluation of Mutual Information Estimators for Time Series

TL;DR

This paper compares various mutual information estimators for time series, analyzing their consistency, parameter sensitivity, and accuracy in different system types and noise conditions, providing guidance for optimal estimator selection.

Contribution

It offers a comprehensive evaluation of mutual information estimators, introducing a data adaptive criterion for binning, and identifies the most stable and accurate methods for different scenarios.

Findings

k-nearest neighbor estimator is most stable and less affected by parameters.

Binning and kernel estimators best identify the first minimum of mutual information in nonlinear systems.

Data adaptive binning criterion is effective for linear systems but conservative for nonlinear systems.

Abstract

We study some of the most commonly used mutual information estimators, based on histograms of fixed or adaptive bin size, $k$-nearest neighbors and kernels, and focus on optimal selection of their free parameters. We examine the consistency of the estimators (convergence to a stable value with the increase of time series length) and the degree of deviation among the estimators. The optimization of parameters is assessed by quantifying the deviation of the estimated mutual information from its true or asymptotic value as a function of the free parameter. Moreover, some common-used criteria for parameter selection are evaluated for each estimator. The comparative study is based on Monte Carlo simulations on time series from several linear and nonlinear systems of different lengths and noise levels. The results show that the $k$-nearest neighbor is the most stable and less affected by the method-specific parameter. A data adaptive criterion for optimal binning is suggested for linear systems but it is found to be rather conservative for nonlinear systems. It turns out that the binning and kernel estimators give the least deviation in identifying the lag of the first minimum of mutual information from nonlinear systems, and are stable in the presence of noise.