Triple-View Knowledge Distillation for Semi-Supervised Semantic Segmentation

TL;DR

This paper introduces TriKD, a semi-supervised semantic segmentation framework using three diverse encoders and a dual-frequency decoder to improve feature diversity and reduce redundancy, achieving better accuracy and efficiency.

Contribution

The paper proposes a novel triple-view encoder with different architectures and a dual-frequency decoder with attention, enhancing feature diversity and reducing memory costs in semi-supervised segmentation.

Findings

Outperforms existing methods on Pascal VOC 2012 and Cityscapes benchmarks.

Achieves a good balance between segmentation accuracy and inference speed.

Effectively utilizes diverse features through triple-view distillation and frequency domain attention.

Abstract

To alleviate the expensive human labeling, semi-supervised semantic segmentation employs a few labeled images and an abundant of unlabeled images to predict the pixel-level label map with the same size. Previous methods often adopt co-training using two convolutional networks with the same architecture but different initialization, which fails to capture the sufficiently diverse features. This motivates us to use tri-training and develop the triple-view encoder to utilize the encoders with different architectures to derive diverse features, and exploit the knowledge distillation skill to learn the complementary semantics among these encoders. Moreover, existing methods simply concatenate the features from both encoder and decoder, resulting in redundant features that require large memory cost. This inspires us to devise a dual-frequency decoder that selects those important features by projecting the features from the spatial domain to the frequency domain, where the dual-frequency channel attention mechanism is introduced to model the feature importance. Therefore, we propose a Triple-view Knowledge Distillation framework, termed TriKD, for semi-supervised semantic segmentation, including the triple-view encoder and the dual-frequency decoder. Extensive experiments were conducted on two benchmarks, \ie, Pascal VOC 2012 and Cityscapes, whose results verify the superiority of the proposed method with a good tradeoff between precision and inference speed.