Scribble-Supervised Semantic Segmentation by Uncertainty Reduction on Neural Representation and Self-Supervision on Neural Eigenspace

TL;DR

This paper introduces a novel scribble-supervised semantic segmentation method that reduces uncertainty through neural representation and self-supervision, achieving performance comparable to fully supervised methods without requiring extensive annotations.

Contribution

It proposes a unified approach combining entropy minimization, neural random walk, and self-supervision on neural eigenspace to improve scribble-based segmentation accuracy.

Findings

Outperforms existing scribble-supervised methods

Comparable to some fully supervised approaches

Effective even with sparse or dropped scribbles

Abstract

Scribble-supervised semantic segmentation has gained much attention recently for its promising performance without high-quality annotations. Due to the lack of supervision, confident and consistent predictions are usually hard to obtain. Typically, people handle these problems to either adopt an auxiliary task with the well-labeled dataset or incorporate the graphical model with additional requirements on scribble annotations. Instead, this work aims to achieve semantic segmentation by scribble annotations directly without extra information and other limitations. Specifically, we propose holistic operations, including minimizing entropy and a network embedded random walk on neural representation to reduce uncertainty. Given the probabilistic transition matrix of a random walk, we further train the network with self-supervision on its neural eigenspace to impose consistency on predictions between related images. Comprehensive experiments and ablation studies verify the proposed approach, which demonstrates superiority over others; it is even comparable to some full-label supervised ones and works well when scribbles are randomly shrunk or dropped.