Scribbles for All: Benchmarking Scribble Supervised Segmentation Across Datasets

TL;DR

This paper introduces Scribbles for All, a method to generate scribble labels for various datasets, enabling improved weakly supervised semantic segmentation research and benchmarking.

Contribution

It provides a novel algorithm to automatically generate scribble labels for any dataset, facilitating new research and evaluation in scribble-based segmentation.

Findings

Models trained on synthetic scribbles perform comparably to those trained on manual labels.

New datasets and benchmarks for scribble segmentation are now available.

The approach accelerates research in weakly supervised semantic segmentation.

Abstract

In this work, we introduce Scribbles for All, a label and training data generation algorithm for semantic segmentation trained on scribble labels. Training or fine-tuning semantic segmentation models with weak supervision has become an important topic recently and was subject to significant advances in model quality. In this setting, scribbles are a promising label type to achieve high quality segmentation results while requiring a much lower annotation effort than usual pixel-wise dense semantic segmentation annotations. The main limitation of scribbles as source for weak supervision is the lack of challenging datasets for scribble segmentation, which hinders the development of novel methods and conclusive evaluations. To overcome this limitation, Scribbles for All provides scribble labels for several popular segmentation datasets and provides an algorithm to automatically generate scribble labels for any dataset with dense annotations, paving the way for new insights and model advancements in the field of weakly supervised segmentation. In addition to providing datasets and algorithm, we evaluate state-of-the-art segmentation models on our datasets and show that models trained with our synthetic labels perform competitively with respect to models trained on manual labels. Thus, our datasets enable state-of-the-art research into methods for scribble-labeled semantic segmentation. The datasets, scribble generation algorithm, and baselines are publicly available at https://github.com/wbkit/Scribbles4All