NP-SemiSeg: When Neural Processes meet Semi-Supervised Semantic Segmentation

TL;DR

This paper introduces NP-SemiSeg, a novel model combining Neural Processes with semi-supervised semantic segmentation, improving uncertainty quantification and segmentation accuracy on benchmarks like PASCAL VOC 2012 and Cityscapes.

Contribution

It adapts Neural Processes to semi-supervised semantic segmentation, providing a new approach for uncertainty estimation and improved segmentation performance.

Findings

NP-SemiSeg outperforms baseline methods on benchmark datasets.

The model effectively quantifies uncertainty in segmentation tasks.

Experimental results demonstrate robustness across different training settings.

Abstract

Semi-supervised semantic segmentation involves assigning pixel-wise labels to unlabeled images at training time. This is useful in a wide range of real-world applications where collecting pixel-wise labels is not feasible in time or cost. Current approaches to semi-supervised semantic segmentation work by predicting pseudo-labels for each pixel from a class-wise probability distribution output by a model. If the predicted probability distribution is incorrect, however, this leads to poor segmentation results, which can have knock-on consequences in safety critical systems, like medical images or self-driving cars. It is, therefore, important to understand what a model does not know, which is mainly achieved by uncertainty quantification. Recently, neural processes (NPs) have been explored in semi-supervised image classification, and they have been a computationally efficient and effective method for uncertainty quantification. In this work, we move one step forward by adapting NPs to semi-supervised semantic segmentation, resulting in a new model called NP-SemiSeg. We experimentally evaluated NP-SemiSeg on the public benchmarks PASCAL VOC 2012 and Cityscapes, with different training settings, and the results verify its effectiveness.