Learning Pixel-wise Labeling from the Internet without Human Interaction

TL;DR

This paper introduces a novel approach for pixel-wise semantic segmentation using only noisy image-level labels from Internet data, eliminating the need for manual annotation and achieving state-of-the-art results.

Contribution

It proposes a class-specific attention model for initial segmentation from noisy data and an online fine-tuning process, pioneering Internet-based weakly supervised segmentation.

Findings

Achieves state-of-the-art performance on PASCAL VOC2012

Demonstrates effectiveness of Internet data for segmentation

Provides a new baseline for Internet-based weak supervision

Abstract

Deep learning stands at the forefront in many computer vision tasks. However, deep neural networks are usually data-hungry and require a huge amount of well-annotated training samples. Collecting sufficient annotated data is very expensive in many applications, especially for pixel-level prediction tasks such as semantic segmentation. To solve this fundamental issue, we consider a new challenging vision task, Internetly supervised semantic segmentation, which only uses Internet data with noisy image-level supervision of corresponding query keywords for segmentation model training. We address this task by proposing the following solution. A class-specific attention model unifying multiscale forward and backward convolutional features is proposed to provide initial segmentation "ground truth". The model trained with such noisy annotations is then improved by an online fine-tuning procedure. It achieves state-of-the-art performance under the weakly-supervised setting on PASCAL VOC2012 dataset. The proposed framework also paves a new way towards learning from the Internet without human interaction and could serve as a strong baseline therein. Code and data will be released upon the paper acceptance.