A Comprehensive Analysis of Weakly-Supervised Semantic Segmentation in Different Image Domains

TL;DR

This paper evaluates the effectiveness of weakly-supervised semantic segmentation methods across different image domains, highlighting domain-specific challenges and the need for more generalizable solutions.

Contribution

It provides a comprehensive cross-domain analysis of existing methods and offers practical techniques for applying them to unseen datasets.

Findings

Methods perform well on their original datasets

Performance drops significantly on histopathology and satellite images

Domain-specific challenges require tailored approaches

Abstract

Recently proposed methods for weakly-supervised semantic segmentation have achieved impressive performance in predicting pixel classes despite being trained with only image labels which lack positional information. Because image annotations are cheaper and quicker to generate, weak supervision is more practical than full supervision for training segmentation algorithms. These methods have been predominantly developed to solve the background separation and partial segmentation problems presented by natural scene images and it is unclear whether they can be simply transferred to other domains with different characteristics, such as histopathology and satellite images, and still perform well. This paper evaluates state-of-the-art weakly-supervised semantic segmentation methods on natural scene, histopathology, and satellite image datasets and analyzes how to determine which method is most suitable for a given dataset. Our experiments indicate that histopathology and satellite images present a different set of problems for weakly-supervised semantic segmentation than natural scene images, such as ambiguous boundaries and class co-occurrence. Methods perform well for datasets they were developed on, but tend to perform poorly on other datasets. We present some practical techniques for these methods on unseen datasets and argue that more work is needed for a generalizable approach to weakly-supervised semantic segmentation. Our full code implementation is available on GitHub: https://github.com/lyndonchan/wsss-analysis.