Efficient Multi-task Uncertainties for Joint Semantic Segmentation and Monocular Depth Estimation

TL;DR

This paper introduces EMUFormer, a novel efficient distillation method that improves joint semantic segmentation and monocular depth estimation, providing high-quality uncertainty estimates with less computational cost.

Contribution

It presents EMUFormer, a new student-teacher distillation approach that enhances multi-task uncertainty quantification and achieves state-of-the-art results in joint semantic segmentation and depth estimation.

Findings

EMUFormer outperforms existing methods in Cityscapes and NYUv2 datasets.

It provides high-quality predictive uncertainties comparable to Deep Ensembles.

The approach is significantly more efficient than traditional ensemble methods.

Abstract

Quantifying the predictive uncertainty emerged as a possible solution to common challenges like overconfidence or lack of explainability and robustness of deep neural networks, albeit one that is often computationally expensive. Many real-world applications are multi-modal in nature and hence benefit from multi-task learning. In autonomous driving, for example, the joint solution of semantic segmentation and monocular depth estimation has proven to be valuable. In this work, we first combine different uncertainty quantification methods with joint semantic segmentation and monocular depth estimation and evaluate how they perform in comparison to each other. Additionally, we reveal the benefits of multi-task learning with regard to the uncertainty quality compared to solving both tasks separately. Based on these insights, we introduce EMUFormer, a novel student-teacher distillation approach for joint semantic segmentation and monocular depth estimation as well as efficient multi-task uncertainty quantification. By implicitly leveraging the predictive uncertainties of the teacher, EMUFormer achieves new state-of-the-art results on Cityscapes and NYUv2 and additionally estimates high-quality predictive uncertainties for both tasks that are comparable or superior to a Deep Ensemble despite being an order of magnitude more efficient.