Uncertainty Estimation for Super-Resolution using ESRGAN

TL;DR

This paper introduces methods to estimate predictive uncertainty in ESRGAN-based super-resolution models, enabling users to identify uncertain pixels without sacrificing image quality.

Contribution

It applies Monte Carlo-Dropout and Deep Ensemble techniques to ESRGAN, providing calibrated uncertainty estimates for super-resolution outputs.

Findings

Uncertainty estimates are decently calibrated.

No performance drop when adding uncertainty estimation.

Enhances ESRGAN with uncertainty quantification methods.

Abstract

Deep Learning-based image super-resolution (SR) has been gaining traction with the aid of Generative Adversarial Networks. Models like SRGAN and ESRGAN are constantly ranked between the best image SR tools. However, they lack principled ways for estimating predictive uncertainty. In the present work, we enhance these models using Monte Carlo-Dropout and Deep Ensemble, allowing the computation of predictive uncertainty. When coupled with a prediction, uncertainty estimates can provide more information to the model users, highlighting pixels where the SR output might be uncertain, hence potentially inaccurate, if these estimates were to be reliable. Our findings suggest that these uncertainty estimates are decently calibrated and can hence fulfill this goal, while providing no performance drop with respect to the corresponding models without uncertainty estimation.