DEAL: Difficulty-aware Active Learning for Semantic Segmentation

TL;DR

This paper introduces DEAL, a difficulty-aware active learning framework for semantic segmentation that focuses on hard semantic areas, leading to improved performance on challenging regions.

Contribution

DEAL is the first to incorporate semantic difficulty scores into active learning for segmentation, enhancing sample selection for hard areas.

Findings

Achieves state-of-the-art active learning performance on benchmarks.

Significantly improves segmentation quality on difficult semantic regions.

Outperforms existing methods in selecting informative samples.

Abstract

Active learning aims to address the paucity of labeled data by finding the most informative samples. However, when applying to semantic segmentation, existing methods ignore the segmentation difficulty of different semantic areas, which leads to poor performance on those hard semantic areas such as tiny or slender objects. To deal with this problem, we propose a semantic Difficulty-awarE Active Learning (DEAL) network composed of two branches: the common segmentation branch and the semantic difficulty branch. For the latter branch, with the supervision of segmentation error between the segmentation result and GT, a pixel-wise probability attention module is introduced to learn the semantic difficulty scores for different semantic areas. Finally, two acquisition functions are devised to select the most valuable samples with semantic difficulty. Competitive results on semantic segmentation benchmarks demonstrate that DEAL achieves state-of-the-art active learning performance and improves the performance of the hard semantic areas in particular.