Gated CRF Loss for Weakly Supervised Semantic Image Segmentation

TL;DR

This paper introduces a Gated CRF loss function for weakly supervised semantic segmentation that improves boundary accuracy and flexibility, achieving state-of-the-art results without relying on complex filtering techniques.

Contribution

The paper proposes a novel Gated CRF loss that enhances weakly supervised segmentation by focusing on semantic boundaries and flexible kernel construction, simplifying implementation.

Findings

Achieves state-of-the-art performance on click-based annotations.

Effective in boundary delineation without high-dimensional filtering.

Flexible kernel design improves contextual relation modeling.

Abstract

State-of-the-art approaches for semantic segmentation rely on deep convolutional neural networks trained on fully annotated datasets, that have been shown to be notoriously expensive to collect, both in terms of time and money. To remedy this situation, weakly supervised methods leverage other forms of supervision that require substantially less annotation effort, but they typically present an inability to predict precise object boundaries due to approximate nature of the supervisory signals in those regions. While great progress has been made in improving the performance, many of these weakly supervised methods are highly tailored to their own specific settings. This raises challenges in reusing algorithms and making steady progress. In this paper, we intentionally avoid such practices when tackling weakly supervised semantic segmentation. In particular, we train standard neural networks with partial cross-entropy loss function for the labeled pixels and our proposed Gated CRF loss for the unlabeled pixels. The Gated CRF loss is designed to deliver several important assets: 1) it enables flexibility in the kernel construction to mask out influence from undesired pixel positions; 2) it offloads learning contextual relations to CNN and concentrates on semantic boundaries; 3) it does not rely on high-dimensional filtering and thus has a simple implementation. Throughout the paper we present the advantages of the loss function, analyze several aspects of weakly supervised training, and show that our `purist' approach achieves state-of-the-art performance for both click-based and scribble-based annotations.