Generalizing Geometric Partition Entropy for the Estimation of Mutual Information in the Presence of Informative Outliers

TL;DR

This paper extends geometric partition entropy to higher dimensions, introduces efficient approximation schemes for mutual information estimation in the presence of informative outliers, and broadens the applicability of entropy-based data analysis.

Contribution

It provides a generalized definition of geometric partition entropy for multi-dimensional data and develops fast approximation methods for mutual information estimation.

Findings

Efficient approximation schemes outperform existing methods in speed.

The new measure-based approach effectively incorporates informative outliers.

Application to chaotic systems demonstrates practical utility.

Abstract

The recent introduction of geometric partition entropy brought a new viewpoint to non-parametric entropy quantification that incorporated the impacts of informative outliers, but its original formulation was limited to the context of a one-dimensional state space. A generalized definition of geometric partition entropy is now provided for samples within a bounded (finite measure) region of a d-dimensional vector space. The basic definition invokes the concept of a Voronoi diagram, but the computational complexity and reliability of Voronoi diagrams in high dimension make estimation by direct theoretical computation unreasonable. This leads to the development of approximation schemes that enable estimation that is faster than current methods by orders of magnitude. The partition intersection ($\pi$) approximation, in particular, enables direct estimates of marginal entropy in any context resulting in an efficient and versatile mutual information estimator. This new measure-based paradigm for data driven information theory allows flexibility in the incorporation of geometry to vary the representation of outlier impact, which leads to a significant broadening in the applicability of established entropy-based concepts. The incorporation of informative outliers is illustrated through analysis of transient dynamics in the synchronization of coupled chaotic dynamical systems.