Learning to Detect Semantic Boundaries with Image-level Class Labels

TL;DR

This paper introduces a novel approach for semantic boundary detection using only image-level class labels, leveraging attention mechanisms and multiple instance learning to generate pseudo labels for training.

Contribution

It proposes a new MIL-based framework and neural network architecture that learn semantic boundaries with weak supervision from image-level labels.

Findings

Achieves competitive performance on SBD dataset

Generates effective pseudo boundary labels from image-level supervision

Outperforms some methods with stronger supervision

Abstract

This paper presents the first attempt to learn semantic boundary detection using image-level class labels as supervision. Our method starts by estimating coarse areas of object classes through attentions drawn by an image classification network. Since boundaries will locate somewhere between such areas of different classes, our task is formulated as a multiple instance learning (MIL) problem, where pixels on a line segment connecting areas of two different classes are regarded as a bag of boundary candidates. Moreover, we design a new neural network architecture that can learn to estimate semantic boundaries reliably even with uncertain supervision given by the MIL strategy. Our network is used to generate pseudo semantic boundary labels of training images, which are in turn used to train fully supervised models. The final model trained with our pseudo labels achieves an outstanding performance on the SBD dataset, where it is as competitive as some of previous arts trained with stronger supervision.