Enabling Uncertainty Estimation in Iterative Neural Networks

TL;DR

This paper introduces a novel method for uncertainty estimation in iterative neural networks by leveraging their convergence rate as a proxy, achieving state-of-the-art results with lower computational cost.

Contribution

It proposes using the convergence rate of iterative neural networks as an uncertainty measure, avoiding modifications to the original model and outperforming ensemble methods.

Findings

Effective uncertainty estimation in two application domains

Lower computational cost compared to ensemble techniques

State-of-the-art accuracy in uncertainty quantification

Abstract

Turning pass-through network architectures into iterative ones, which use their own output as input, is a well-known approach for boosting performance. In this paper, we argue that such architectures offer an additional benefit: The convergence rate of their successive outputs is highly correlated with the accuracy of the value to which they converge. Thus, we can use the convergence rate as a useful proxy for uncertainty. This results in an approach to uncertainty estimation that provides state-of-the-art estimates at a much lower computational cost than techniques like Ensembles, and without requiring any modifications to the original iterative model. We demonstrate its practical value by embedding it in two application domains: road detection in aerial images and the estimation of aerodynamic properties of 2D and 3D shapes.