Progressive Learning with Cross-Window Consistency for Semi-Supervised Semantic Segmentation

TL;DR

This paper introduces a novel semi-supervised semantic segmentation framework leveraging cross-window consistency and progressive learning, significantly improving performance by effectively utilizing unlabeled data.

Contribution

It proposes a cross-window consistency-driven progressive learning framework with a biased loss and dynamic pseudo-label memory bank for better unlabeled data exploitation.

Findings

Outperforms state-of-the-art methods on urban, medical, and satellite datasets.

Effectively leverages unlabeled data through cross-window consistency.

Achieves significant accuracy improvements across diverse scenarios.

Abstract

Semi-supervised semantic segmentation focuses on the exploration of a small amount of labeled data and a large amount of unlabeled data, which is more in line with the demands of real-world image understanding applications. However, it is still hindered by the inability to fully and effectively leverage unlabeled images. In this paper, we reveal that cross-window consistency (CWC) is helpful in comprehensively extracting auxiliary supervision from unlabeled data. Additionally, we propose a novel CWC-driven progressive learning framework to optimize the deep network by mining weak-to-strong constraints from massive unlabeled data. More specifically, this paper presents a biased cross-window consistency (BCC) loss with an importance factor, which helps the deep network explicitly constrain confidence maps from overlapping regions in different windows to maintain semantic consistency with larger contexts. In addition, we propose a dynamic pseudo-label memory bank (DPM) to provide high-consistency and high-reliability pseudo-labels to further optimize the network. Extensive experiments on three representative datasets of urban views, medical scenarios, and satellite scenes demonstrate our framework consistently outperforms the state-of-the-art methods with a large margin. Code will be available publicly.