Semantic uncertainty intervals for disentangled latent spaces

TL;DR

This paper introduces a method to generate principled, calibrated uncertainty intervals for semantic factors in disentangled latent spaces, enhancing interpretability and reliability in computer vision tasks.

Contribution

It proposes a novel approach combining quantile regression and calibration to produce semantic uncertainty intervals that are guaranteed to contain true factors across generative models.

Findings

Uncertainty intervals reliably contain true semantic factors.

Method improves interpretability of generative model outputs.

Applicable to inverse problems like super-resolution and image completion.

Abstract

Meaningful uncertainty quantification in computer vision requires reasoning about semantic information -- say, the hair color of the person in a photo or the location of a car on the street. To this end, recent breakthroughs in generative modeling allow us to represent semantic information in disentangled latent spaces, but providing uncertainties on the semantic latent variables has remained challenging. In this work, we provide principled uncertainty intervals that are guaranteed to contain the true semantic factors for any underlying generative model. The method does the following: (1) it uses quantile regression to output a heuristic uncertainty interval for each element in the latent space (2) calibrates these uncertainties such that they contain the true value of the latent for a new, unseen input. The endpoints of these calibrated intervals can then be propagated through the generator to produce interpretable uncertainty visualizations for each semantic factor. This technique reliably communicates semantically meaningful, principled, and instance-adaptive uncertainty in inverse problems like image super-resolution and image completion.