Learning Semantic Segmentation with Diverse Supervision

TL;DR

This paper introduces a flexible approach for training semantic segmentation models using various types of annotations, reducing reliance on costly pixel-level labels and leveraging diverse data sources.

Contribution

It presents a novel method that integrates different supervision levels into CNN training for semantic segmentation, enhancing performance with less annotation effort.

Findings

Effective use of diverse annotations improves segmentation accuracy.

Method outperforms traditional approaches on PASCAL VOC 2012 and SIFT-flow.

Compatible with existing CNN-based segmentation networks.

Abstract

Models based on deep convolutional neural networks (CNN) have significantly improved the performance of semantic segmentation. However, learning these models requires a large amount of training images with pixel-level labels, which are very costly and time-consuming to collect. In this paper, we propose a method for learning CNN-based semantic segmentation models from images with several types of annotations that are available for various computer vision tasks, including image-level labels for classification, box-level labels for object detection and pixel-level labels for semantic segmentation. The proposed method is flexible and can be used together with any existing CNN-based semantic segmentation networks. Experimental evaluation on the challenging PASCAL VOC 2012 and SIFT-flow benchmarks demonstrate that the proposed method can effectively make use of diverse training data to improve the performance of the learned models.