Boosting Semi-Supervised Semantic Segmentation with Probabilistic Representations

TL;DR

This paper introduces a probabilistic contrastive learning framework for semi-supervised semantic segmentation that models pixel representations as probability distributions, improving robustness against pseudo-label inaccuracies and enhancing segmentation performance.

Contribution

It proposes a novel probabilistic representation contrastive learning method that models pixel features as Gaussian distributions and regularizes their variance, advancing semi-supervised segmentation.

Findings

Outperforms existing methods on Pascal VOC and CityScapes datasets.

Models pixel representations as Gaussian distributions for better uncertainty handling.

Regularizing distribution variance improves segmentation accuracy.

Abstract

Recent breakthroughs in semi-supervised semantic segmentation have been developed through contrastive learning. In prevalent pixel-wise contrastive learning solutions, the model maps pixels to deterministic representations and regularizes them in the latent space. However, there exist inaccurate pseudo-labels which map the ambiguous representations of pixels to the wrong classes due to the limited cognitive ability of the model. In this paper, we define pixel-wise representations from a new perspective of probability theory and propose a Probabilistic Representation Contrastive Learning (PRCL) framework that improves representation quality by taking its probability into consideration. Through modelling the mapping from pixels to representations as the probability via multivariate Gaussian distributions, we can tune the contribution of the ambiguous representations to tolerate the risk of inaccurate pseudo-labels. Furthermore, we define prototypes in the form of distributions, which indicates the confidence of a class, while the point prototype cannot. Moreover, we propose to regularize the distribution variance to enhance the reliability of representations. Taking advantage of these benefits, high-quality feature representations can be derived in the latent space, thereby the performance of semantic segmentation can be further improved. We conduct sufficient experiment to evaluate PRCL on Pascal VOC and CityScapes to demonstrate its superiority. The code is available at https://github.com/Haoyu-Xie/PRCL.