Branches Mutual Promotion for End-to-End Weakly Supervised Semantic Segmentation

TL;DR

This paper introduces a mutual promotion framework for weakly supervised semantic segmentation, enabling classification and segmentation branches to enhance each other through bidirectional supervision and interaction operations, leading to improved performance.

Contribution

It proposes a novel mutual promotion approach with bidirectional supervision and interaction operations, balancing the roles of classification and segmentation branches in end-to-end training.

Findings

Outperforms existing weakly supervised segmentation methods

Enhances localization seed quality through feedback mechanisms

Achieves better segmentation accuracy in experiments

Abstract

End-to-end weakly supervised semantic segmentation aims at optimizing a segmentation model in a single-stage training process based on only image annotations. Existing methods adopt an online-trained classification branch to provide pseudo annotations for supervising the segmentation branch. However, this strategy makes the classification branch dominate the whole concurrent training process, hindering these two branches from assisting each other. In our work, we treat these two branches equally by viewing them as diverse ways to generate the segmentation map, and add interactions on both their supervision and operation to achieve mutual promotion. For this purpose, a bidirectional supervision mechanism is elaborated to force the consistency between the outputs of these two branches. Thus, the segmentation branch can also give feedback to the classification branch to enhance the quality of localization seeds. Moreover, our method also designs interaction operations between these two branches to exchange their knowledge to assist each other. Experiments indicate our work outperforms existing end-to-end weakly supervised segmentation methods.