Information Theoretical Estimators Toolbox

TL;DR

The paper introduces ITE, a versatile, open-source toolbox for estimating various information-theoretic quantities, supporting modular design, customization, and application in signal processing tasks.

Contribution

It provides a highly modular, open-source toolbox for estimating diverse information-theoretic measures, enabling easy customization and application in optimization problems.

Findings

Supports multiple variants of entropy, mutual information, divergence, and kernels.

Facilitates combination and embedding of new estimators.

Includes a prototype application in signal processing, independent subspace analysis.

Abstract

We present ITE (information theoretical estimators) a free and open source, multi-platform, Matlab/Octave toolbox that is capable of estimating many different variants of entropy, mutual information, divergence, association measures, cross quantities, and kernels on distributions. Thanks to its highly modular design, ITE supports additionally (i) the combinations of the estimation techniques, (ii) the easy construction and embedding of novel information theoretical estimators, and (iii) their immediate application in information theoretical optimization problems. ITE also includes a prototype application in a central problem class of signal processing, independent subspace analysis and its extensions.