Fully Convolutional Multi-Class Multiple Instance Learning

TL;DR

This paper introduces a fully convolutional multi-class MIL approach for semantic segmentation that learns from weak image-level labels, enabling end-to-end training without object proposals.

Contribution

It presents a novel multi-class MIL formulation with a fully convolutional network that jointly optimizes segmentation and label disambiguation from weak supervision.

Findings

Effective on PASCAL VOC dataset

No need for object proposal pre-processing

Supports inputs of any size

Abstract

Multiple instance learning (MIL) can reduce the need for costly annotation in tasks such as semantic segmentation by weakening the required degree of supervision. We propose a novel MIL formulation of multi-class semantic segmentation learning by a fully convolutional network. In this setting, we seek to learn a semantic segmentation model from just weak image-level labels. The model is trained end-to-end to jointly optimize the representation while disambiguating the pixel-image label assignment. Fully convolutional training accepts inputs of any size, does not need object proposal pre-processing, and offers a pixelwise loss map for selecting latent instances. Our multi-class MIL loss exploits the further supervision given by images with multiple labels. We evaluate this approach through preliminary experiments on the PASCAL VOC segmentation challenge.