Semantic-Rearrangement-Based Multi-Level Alignment for Domain Generalized Segmentation

TL;DR

This paper introduces SRMA, a novel method for domain generalized semantic segmentation that uses semantic rearrangement and multi-level alignment to improve domain invariance and performance across unseen target domains.

Contribution

The paper proposes SRMA, combining semantic region randomization and multi-level feature alignment to better capture regional discrepancies and enhance domain generalization in segmentation.

Findings

SRMA outperforms state-of-the-art methods on multiple benchmarks.

Semantic rearrangement enhances source domain diversity effectively.

Multi-level alignment improves domain-invariant feature learning.

Abstract

Domain generalized semantic segmentation is an essential computer vision task, for which models only leverage source data to learn the capability of generalized semantic segmentation towards the unseen target domains. Previous works typically address this challenge by global style randomization or feature regularization. In this paper, we argue that given the observation that different local semantic regions perform different visual characteristics from the source domain to the target domain, methods focusing on global operations are hard to capture such regional discrepancies, thus failing to construct domain-invariant representations with the consistency from local to global level. Therefore, we propose the Semantic-Rearrangement-based Multi-Level Alignment (SRMA) to overcome this problem. SRMA first incorporates a Semantic Rearrangement Module (SRM), which conducts semantic region randomization to enhance the diversity of the source domain sufficiently. A Multi-Level Alignment module (MLA) is subsequently proposed with the help of such diversity to establish the global-regional-local consistent domain-invariant representations. By aligning features across randomized samples with domain-neutral knowledge at multiple levels, SRMA provides a more robust way to handle the source-target domain gap. Extensive experiments demonstrate the superiority of SRMA over the current state-of-the-art works on various benchmarks.