Towards Uncertainty Quantification in Generative Model Learning

TL;DR

This paper formalizes the challenge of quantifying uncertainty in generative models, proposing ensemble-based methods and preliminary experiments to evaluate model approximation uncertainty systematically.

Contribution

It introduces the formal problem of uncertainty quantification in generative models and explores ensemble-based evaluation methods with initial experimental validation.

Findings

Aggregated precision-recall curves effectively capture model uncertainty.

Preliminary experiments demonstrate the potential of these methods on synthetic data.

Systematic comparison of generative models based on uncertainty characteristics is feasible.

Abstract

While generative models have become increasingly prevalent across various domains, fundamental concerns regarding their reliability persist. A crucial yet understudied aspect of these models is the uncertainty quantification surrounding their distribution approximation capabilities. Current evaluation methodologies focus predominantly on measuring the closeness between the learned and the target distributions, neglecting the inherent uncertainty in these measurements. In this position paper, we formalize the problem of uncertainty quantification in generative model learning. We discuss potential research directions, including the use of ensemble-based precision-recall curves. Our preliminary experiments on synthetic datasets demonstrate the effectiveness of aggregated precision-recall curves in capturing model approximation uncertainty, enabling systematic comparison among different model architectures based on their uncertainty characteristics.