ViewAL: Active Learning with Viewpoint Entropy for Semantic Segmentation

TL;DR

ViewAL introduces a viewpoint entropy-based active learning strategy for semantic segmentation that leverages multi-view consistency and superpixel uncertainty to reduce annotation effort while maintaining high performance.

Contribution

The paper presents a novel active learning method using viewpoint entropy and superpixel uncertainty, improving data efficiency in semantic segmentation tasks.

Findings

Achieves 95% of maximum performance with significantly less labeled data.

Reduces labeling effort by 25% when using superpixels.

Outperforms state-of-the-art methods in data efficiency.

Abstract

We propose ViewAL, a novel active learning strategy for semantic segmentation that exploits viewpoint consistency in multi-view datasets. Our core idea is that inconsistencies in model predictions across viewpoints provide a very reliable measure of uncertainty and encourage the model to perform well irrespective of the viewpoint under which objects are observed. To incorporate this uncertainty measure, we introduce a new viewpoint entropy formulation, which is the basis of our active learning strategy. In addition, we propose uncertainty computations on a superpixel level, which exploits inherently localized signal in the segmentation task, directly lowering the annotation costs. This combination of viewpoint entropy and the use of superpixels allows to efficiently select samples that are highly informative for improving the network. We demonstrate that our proposed active learning strategy not only yields the best-performing models for the same amount of required labeled data, but also significantly reduces labeling effort. For instance, our method achieves 95% of maximum achievable network performance using only 7%, 17%, and 24% labeled data on SceneNet-RGBD, ScanNet, and Matterport3D, respectively. On these datasets, the best state-of-the-art method achieves the same performance with 14%, 27% and 33% labeled data. Finally, we demonstrate that labeling using superpixels yields the same quality of ground-truth compared to labeling whole images, but requires 25% less time.