Semantic Segmentation Refinement by Monte Carlo Region Growing of High Confidence Detections

TL;DR

This paper introduces a Monte Carlo region growing technique to refine semantic segmentation results by re-labeling low-confidence pixels, significantly improving boundary adherence and accuracy on multiple datasets.

Contribution

The proposed method enhances segmentation accuracy by refining deep learning outputs through a Monte Carlo region growing process based on confidence scores.

Findings

Improved boundary adherence in segmentation results.

Significant accuracy gains on COCO and PASCAL datasets.

Better boundary refinement on DAVIS sequences.

Abstract

Despite recent improvements using fully convolutional networks, in general, the segmentation produced by most state-of-the-art semantic segmentation methods does not show satisfactory adherence to the object boundaries. We propose a method to refine the segmentation results generated by such deep learning models. Our method takes as input the confidence scores generated by a pixel-dense segmentation network and re-labels pixels with low confidence levels. The re-labeling approach employs a region growing mechanism that aggregates these pixels to neighboring areas with high confidence scores and similar appearance. In order to correct the labels of pixels that were incorrectly classified with high confidence level by the semantic segmentation algorithm, we generate multiple region growing steps through a Monte Carlo sampling of the seeds of the regions. Our method improves the accuracy of a state-of-the-art fully convolutional semantic segmentation approach on the publicly available COCO and PASCAL datasets, and it shows significantly better results on selected sequences of the finely-annotated DAVIS dataset.