Multiple Fusion Adaptation: A Strong Framework for Unsupervised Semantic Segmentation Adaptation

TL;DR

This paper introduces Multiple Fusion Adaptation (MFA), a novel framework for unsupervised domain adaptation in semantic segmentation that combines three fusion strategies to improve accuracy without additional annotations.

Contribution

The paper proposes a unified MFA framework that effectively integrates cross-model, temporal, and online-offline pseudo label fusion strategies for better domain adaptation.

Findings

Significant improvement in segmentation accuracy on benchmarks

Achieved new state-of-the-art results with 58.2% and 62.5% mIoU

Demonstrated the effectiveness of multi-strategy fusion in UDA

Abstract

This paper challenges the cross-domain semantic segmentation task, aiming to improve the segmentation accuracy on the unlabeled target domain without incurring additional annotation. Using the pseudo-label-based unsupervised domain adaptation (UDA) pipeline, we propose a novel and effective Multiple Fusion Adaptation (MFA) method. MFA basically considers three parallel information fusion strategies, i.e., the cross-model fusion, temporal fusion and a novel online-offline pseudo label fusion. Specifically, the online-offline pseudo label fusion encourages the adaptive training to pay additional attention to difficult regions that are easily ignored by offline pseudo labels, therefore retaining more informative details. While the other two fusion strategies may look standard, MFA pays significant efforts to raise the efficiency and effectiveness for integration, and succeeds in injecting all the three strategies into a unified framework. Experiments on two widely used benchmarks, i.e., GTA5-to-Cityscapes and SYNTHIA-to-Cityscapes, show that our method significantly improves the semantic segmentation adaptation, and sets up new state of the art (58.2% and 62.5% mIoU, respectively). The code will be available at https://github.com/KaiiZhang/MFA.