On the pitfalls of entropy-based uncertainty for multi-class semi-supervised segmentation

TL;DR

This paper reveals that entropy-based uncertainty measures are ineffective for multi-class semi-supervised segmentation due to inter-class overlap issues, and proposes a divergence-based alternative that improves performance.

Contribution

The paper identifies limitations of entropy-based uncertainty in multi-class segmentation and introduces a divergence-based method to better handle inter-class overlap.

Findings

Entropy-based uncertainty underperforms in multi-class scenarios.

Divergence-based uncertainty improves segmentation accuracy.

Proposed method enhances existing uncertainty-based segmentation models.

Abstract

Semi-supervised learning has emerged as an appealing strategy to train deep models with limited supervision. Most prior literature under this learning paradigm resorts to dual-based architectures, typically composed of a teacher-student duple. To drive the learning of the student, many of these models leverage the aleatoric uncertainty derived from the entropy of the predictions. While this has shown to work well in a binary scenario, we demonstrate in this work that this strategy leads to suboptimal results in a multi-class context, a more realistic and challenging setting. We argue, indeed, that these approaches underperform due to the erroneous uncertainty approximations in the presence of inter-class overlap. Furthermore, we propose an alternative solution to compute the uncertainty in a multi-class setting, based on divergence distances and which account for inter-class overlap. We evaluate the proposed solution on a challenging multi-class segmentation dataset and in two well-known uncertainty-based segmentation methods. The reported results demonstrate that by simply replacing the mechanism used to compute the uncertainty, our proposed solution brings substantial improvement on tested setups.