A Comparative Study on Multi-task Uncertainty Quantification in Semantic Segmentation and Monocular Depth Estimation

TL;DR

This paper evaluates uncertainty quantification methods in multi-task perception models for autonomous driving, revealing Deep Ensembles as most effective, especially out-of-domain, and highlighting the benefits of multi-task learning for uncertainty quality.

Contribution

It provides a comprehensive comparison of uncertainty quantification techniques in multi-task perception, emphasizing Deep Ensembles and the advantages of joint learning.

Findings

Deep Ensembles outperform other methods in out-of-domain scenarios.

Multi-task learning improves uncertainty estimation quality.

Median uncertainty is a robust threshold for pixel classification.

Abstract

Deep neural networks excel in perception tasks such as semantic segmentation and monocular depth estimation, making them indispensable in safety-critical applications like autonomous driving and industrial inspection. However, they often suffer from overconfidence and poor explainability, especially for out-of-domain data. While uncertainty quantification has emerged as a promising solution to these challenges, multi-task settings have yet to be explored. In an effort to shed light on this, we evaluate Monte Carlo Dropout, Deep Sub-Ensembles, and Deep Ensembles for joint semantic segmentation and monocular depth estimation. Thereby, we reveal that Deep Ensembles stand out as the preferred choice, particularly in out-of-domain scenarios, and show the potential benefit of multi-task learning with regard to the uncertainty quality in comparison to solving both tasks separately. Additionally, we highlight the impact of employing different uncertainty thresholds to classify pixels as certain or uncertain, with the median uncertainty emerging as a robust default.