Learning random-walk label propagation for weakly-supervised semantic segmentation

TL;DR

This paper introduces a novel weakly-supervised semantic segmentation method that propagates sparse labels using random-walk probabilities, enabling effective training of segmentation networks without dense annotations.

Contribution

It presents a differentiable label propagation technique based on random walks, jointly learned with the segmentation model, to improve weakly-supervised semantic segmentation.

Findings

Outperforms naive weakly-supervised methods

Effectively learns semantic edges without direct supervision

Incorporates uncertainty estimates into training

Abstract

Large-scale training for semantic segmentation is challenging due to the expense of obtaining training data for this task relative to other vision tasks. We propose a novel training approach to address this difficulty. Given cheaply-obtained sparse image labelings, we propagate the sparse labels to produce guessed dense labelings. A standard CNN-based segmentation network is trained to mimic these labelings. The label-propagation process is defined via random-walk hitting probabilities, which leads to a differentiable parameterization with uncertainty estimates that are incorporated into our loss. We show that by learning the label-propagator jointly with the segmentation predictor, we are able to effectively learn semantic edges given no direct edge supervision. Experiments also show that training a segmentation network in this way outperforms the naive approach.