On Efficient Uncertainty Estimation for Resource-Constrained Mobile Applications

TL;DR

This paper presents a method to efficiently estimate uncertainty in deep neural networks for mobile applications, balancing prediction quality and computational constraints.

Contribution

It introduces a novel approach based on Monte Carlo Dropout with diversification and branching techniques to improve uncertainty estimation on resource-limited devices.

Findings

Achieves near Deep Ensemble prediction quality

Provides faster inference suitable for mobile platforms

Effective on classification and segmentation tasks

Abstract

Deep neural networks have shown great success in prediction quality while reliable and robust uncertainty estimation remains a challenge. Predictive uncertainty supplements model predictions and enables improved functionality of downstream tasks including embedded and mobile applications, such as virtual reality, augmented reality, sensor fusion, and perception. These applications often require a compromise in complexity to obtain uncertainty estimates due to very limited memory and compute resources. We tackle this problem by building upon Monte Carlo Dropout (MCDO) models using the Axolotl framework; specifically, we diversify sampled subnetworks, leverage dropout patterns, and use a branching technique to improve predictive performance while maintaining fast computations. We conduct experiments on (1) a multi-class classification task using the CIFAR10 dataset, and (2) a more complex human body segmentation task. Our results show the effectiveness of our approach by reaching close to Deep Ensemble prediction quality and uncertainty estimation, while still achieving faster inference on resource-limited mobile platforms.