Simpler Does It: Generating Semantic Labels with Objectness Guidance

TL;DR

This paper introduces a new framework that leverages objectness guidance and weak annotations to generate high-quality pseudo-labels for semantic segmentation, improving performance over existing weakly and semi-supervised methods.

Contribution

The authors propose an end-to-end multi-task learning approach combining objectness and weak annotations to produce better pseudo-labels for semantic segmentation.

Findings

High-quality pseudo-labels generated for unseen categories.

Achieves superior or comparable results to existing weakly/semi-supervised methods.

Effective across various domains with extensive experiments.

Abstract

Existing weakly or semi-supervised semantic segmentation methods utilize image or box-level supervision to generate pseudo-labels for weakly labeled images. However, due to the lack of strong supervision, the generated pseudo-labels are often noisy near the object boundaries, which severely impacts the network's ability to learn strong representations. To address this problem, we present a novel framework that generates pseudo-labels for training images, which are then used to train a segmentation model. To generate pseudo-labels, we combine information from: (i) a class agnostic objectness network that learns to recognize object-like regions, and (ii) either image-level or bounding box annotations. We show the efficacy of our approach by demonstrating how the objectness network can naturally be leveraged to generate object-like regions for unseen categories. We then propose an end-to-end multi-task learning strategy, that jointly learns to segment semantics and objectness using the generated pseudo-labels. Extensive experiments demonstrate the high quality of our generated pseudo-labels and effectiveness of the proposed framework in a variety of domains. Our approach achieves better or competitive performance compared to existing weakly-supervised and semi-supervised methods.