Mutual Information Estimation via Normalizing Flows

TL;DR

This paper introduces a new method for estimating mutual information using normalizing flows, enabling more accurate estimation in high-dimensional data by transforming data to a target distribution with known MI.

Contribution

The paper presents a novel MI estimator based on normalizing flows and provides theoretical guarantees for its accuracy on original data.

Findings

Effective in high-dimensional settings

Theoretically sound with guarantees

Practical advantages demonstrated in experiments

Abstract

We propose a novel approach to the problem of mutual information (MI) estimation via introducing a family of estimators based on normalizing flows. The estimator maps original data to the target distribution, for which MI is easier to estimate. We additionally explore the target distributions with known closed-form expressions for MI. Theoretical guarantees are provided to demonstrate that our approach yields MI estimates for the original data. Experiments with high-dimensional data are conducted to highlight the practical advantages of the proposed method.