MIND: Monge Inception Distance for Generative Models Evaluation

TL;DR

The paper introduces MIND, a new metric for evaluating generative models that is more sample-efficient, faster, and more robust than FID, using sliced Wasserstein distances for improved performance.

Contribution

MIND offers a novel evaluation metric leveraging sliced Wasserstein distance, overcoming FID's limitations in sample complexity, computation speed, and adversarial robustness.

Findings

MIND is ten times more sample-efficient than FID.

MIND is two hundred times faster to compute.

MIND is more robust to adversarial attacks like moment-matching.

Abstract

We propose the Monge Inception Distance (MIND), a metric for evaluating generative models that addresses key limitations of the widely adopted Fr\'echet Inception Distance (FID). The MIND metric leverages the sliced Wasserstein distance to compare distributions by averaging one-dimensional optimal transport distances, efficiently computed via sorting. This approach circumvents the estimation of high-dimensional means and covariance matrices, which underlie FID's poor sample complexity and vulnerability to adversarial attacks. We empirically demonstrate three primary advantages: (i) it is more sample-efficient by one order of magnitude, (ii) it is faster to compute by two orders of magnitude, (iii) it is more robust to adversarial attacks such as moment-matching. We show that MIND with 5k samples can replace the evaluation performance of FID with 50k samples, providing high correlation with this standard benchmark and superior discriminative performance. We further demonstrate that even smaller sample sizes (e.g., 1k or 2k) remain highly informative for rapid model iteration.