FMMI: Flow Matching Mutual Information Estimation

TL;DR

The paper presents FMMI, a new mutual information estimator that uses flow matching to transform distributions, offering efficient and accurate estimates suitable for high-dimensional data.

Contribution

It introduces a flow matching-based MI estimator that improves computational efficiency and accuracy over traditional discriminative methods.

Findings

Provides precise MI estimates across various dimensions

Scales efficiently to high-dimensional data

Outperforms existing estimators in accuracy and speed

Abstract

We introduce a novel Mutual Information (MI) estimator that fundamentally reframes the discriminative approach. Instead of training a classifier to discriminate between joint and marginal distributions, we learn a normalizing flow that transforms one into the other. This technique produces a computationally efficient and precise MI estimate that scales well to high dimensions and across a wide range of ground-truth MI values.