Probabilistic Semantic Segmentation Refinement by Monte Carlo Region Growing

TL;DR

This paper presents pRGR, an unsupervised post-processing method that refines semantic segmentation by propagating labels through Monte Carlo sampling and Bayesian clustering, improving boundary adherence and providing uncertainty estimates.

Contribution

The paper introduces pRGR, a novel probabilistic region growing algorithm that refines segmentation results and estimates uncertainty without supervision, based on Bayesian modeling and Monte Carlo sampling.

Findings

pRGR improves segmentation boundary accuracy across multiple networks.

Uncertainty estimates from pRGR correlate with segmentation accuracy.

The method is effective on various benchmark datasets.

Abstract

Semantic segmentation with fine-grained pixel-level accuracy is a fundamental component of a variety of computer vision applications. However, despite the large improvements provided by recent advances in the architectures of convolutional neural networks, segmentations provided by modern state-of-the-art methods still show limited boundary adherence. We introduce a fully unsupervised post-processing algorithm that exploits Monte Carlo sampling and pixel similarities to propagate high-confidence pixel labels into regions of low-confidence classification. Our algorithm, which we call probabilistic Region Growing Refinement (pRGR), is based on a rigorous mathematical foundation in which clusters are modelled as multivariate normally distributed sets of pixels. Exploiting concepts of Bayesian estimation and variance reduction techniques, pRGR performs multiple refinement iterations at varied receptive fields sizes, while updating cluster statistics to adapt to local image features. Experiments using multiple modern semantic segmentation networks and benchmark datasets demonstrate the effectiveness of our approach for the refinement of segmentation predictions at different levels of coarseness, as well as the suitability of the variance estimates obtained in the Monte Carlo iterations as uncertainty measures that are highly correlated with segmentation accuracy.