Tackling the Problem of Limited Data and Annotations in Semantic Segmentation

TL;DR

This paper investigates improving semantic segmentation on small datasets with limited annotations by leveraging pre-trained models and CRF methods, demonstrating significant performance gains.

Contribution

It introduces the use of SWSL pre-trained models and dense CRF techniques to enhance segmentation accuracy under weak supervision and limited data conditions.

Findings

Pre-trained SWSL models outperform ImageNet pre-trained models by 7.4%.

Dense CRF improves segmentation results as regularizer and post-processing.

Using SWSL pre-training increases mIoU by nearly 4% on full dataset.

Abstract

In this work, the case of semantic segmentation on a small image dataset (simulated by 1000 randomly selected images from PASCAL VOC 2012), where only weak supervision signals (scribbles from user interaction) are available is studied. Especially, to tackle the problem of limited data annotations in image segmentation, transferring different pre-trained models and CRF based methods are applied to enhance the segmentation performance. To this end, RotNet, DeeperCluster, and Semi&Weakly Supervised Learning (SWSL) pre-trained models are transferred and finetuned in a DeepLab-v2 baseline, and dense CRF is applied both as a post-processing and loss regularization technique. The results of my study show that, on this small dataset, using a pre-trained ResNet50 SWSL model gives results that are 7.4% better than applying an ImageNet pre-trained model; moreover, for the case of training on the full PASCAL VOC 2012 training data, this pre-training approach increases the mIoU results by almost 4%. On the other hand, dense CRF is shown to be very effective as well, enhancing the results both as a loss regularization technique in weakly supervised training and as a post-processing tool.