Semantic-Aware Domain Generalized Segmentation

TL;DR

This paper introduces a novel domain generalization framework for semantic segmentation that enhances category discrimination and domain invariance without target domain data, using Semantic-Aware Normalization and Whitening modules.

Contribution

It proposes two new modules, SAN and SAW, to improve feature discrimination and domain invariance in segmentation models trained without target data.

Findings

Significant performance improvements over state-of-the-art methods.

Effective domain generalization across multiple datasets.

Robust segmentation with various backbone networks.

Abstract

Deep models trained on source domain lack generalization when evaluated on unseen target domains with different data distributions. The problem becomes even more pronounced when we have no access to target domain samples for adaptation. In this paper, we address domain generalized semantic segmentation, where a segmentation model is trained to be domain-invariant without using any target domain data. Existing approaches to tackle this problem standardize data into a unified distribution. We argue that while such a standardization promotes global normalization, the resulting features are not discriminative enough to get clear segmentation boundaries. To enhance separation between categories while simultaneously promoting domain invariance, we propose a framework including two novel modules: Semantic-Aware Normalization (SAN) and Semantic-Aware Whitening (SAW). Specifically, SAN focuses on category-level center alignment between features from different image styles, while SAW enforces distributed alignment for the already center-aligned features. With the help of SAN and SAW, we encourage both intra-category compactness and inter-category separability. We validate our approach through extensive experiments on widely-used datasets (i.e. GTAV, SYNTHIA, Cityscapes, Mapillary and BDDS). Our approach shows significant improvements over existing state-of-the-art on various backbone networks. Code is available at https://github.com/leolyj/SAN-SAW