Dual Path Learning for Domain Adaptation of Semantic Segmentation

TL;DR

This paper introduces a dual path learning framework for domain adaptation in semantic segmentation, effectively reducing visual inconsistencies by leveraging two interactive adaptation pipelines for source and target domains.

Contribution

It proposes a novel dual path learning approach with dual path image translation and adaptive segmentation, improving domain adaptation performance in semantic segmentation.

Findings

Outperforms state-of-the-art methods on GTA5→Cityscapes and SYNTHIA→Cityscapes

Utilizes only one segmentation model during inference

Demonstrates effective alleviation of visual inconsistency issues

Abstract

Domain adaptation for semantic segmentation enables to alleviate the need for large-scale pixel-wise annotations. Recently, self-supervised learning (SSL) with a combination of image-to-image translation shows great effectiveness in adaptive segmentation. The most common practice is to perform SSL along with image translation to well align a single domain (the source or target). However, in this single-domain paradigm, unavoidable visual inconsistency raised by image translation may affect subsequent learning. In this paper, based on the observation that domain adaptation frameworks performed in the source and target domain are almost complementary in terms of image translation and SSL, we propose a novel dual path learning (DPL) framework to alleviate visual inconsistency. Concretely, DPL contains two complementary and interactive single-domain adaptation pipelines aligned in source and target domain respectively. The inference of DPL is extremely simple, only one segmentation model in the target domain is employed. Novel technologies such as dual path image translation and dual path adaptive segmentation are proposed to make two paths promote each other in an interactive manner. Experiments on GTA5$\rightarrow$Cityscapes and SYNTHIA$\rightarrow$Cityscapes scenarios demonstrate the superiority of our DPL model over the state-of-the-art methods. The code and models are available at: \url{https://github.com/royee182/DPL}