Learning to Segment via Cut-and-Paste

TL;DR

This paper introduces a weakly-supervised object segmentation method that uses an adversarial cut-and-paste game to learn masks from bounding boxes, achieving near-supervised performance without mask annotations.

Contribution

The novel approach combines adversarial learning with cut-and-paste techniques to improve weakly-supervised segmentation without relying on hand-tuned proposals.

Findings

Outperforms existing weakly supervised methods

Achieves 90% of fully supervised performance

Works across diverse datasets like Cityscapes, COCO, and aerial images

Abstract

This paper presents a weakly-supervised approach to object instance segmentation. Starting with known or predicted object bounding boxes, we learn object masks by playing a game of cut-and-paste in an adversarial learning setup. A mask generator takes a detection box and Faster R-CNN features, and constructs a segmentation mask that is used to cut-and-paste the object into a new image location. The discriminator tries to distinguish between real objects, and those cut and pasted via the generator, giving a learning signal that leads to improved object masks. We verify our method experimentally using Cityscapes, COCO, and aerial image datasets, learning to segment objects without ever having seen a mask in training. Our method exceeds the performance of existing weakly supervised methods, without requiring hand-tuned segment proposals, and reaches 90% of supervised performance.