Learning to Augment: Hallucinating Data for Domain Generalized Segmentation

TL;DR

This paper introduces a GAN-based feature augmentation method that generates diverse stylized features within the source domain to improve domain generalization in semantic segmentation, outperforming existing techniques.

Contribution

The paper proposes GBFA, a novel GAN-based feature augmentation approach that synthesizes stylized features without external data, enhancing domain generalization in segmentation tasks.

Findings

Achieves state-of-the-art results on multiple domain generalization benchmarks.

Effectively generates diverse stylized features within the source domain.

Improves model robustness to unseen target domains.

Abstract

Domain generalized semantic segmentation (DGSS) is an essential but highly challenging task, in which the model is trained only on source data and any target data is not available. Existing DGSS methods primarily standardize the feature distribution or utilize extra domain data for augmentation. However, the former sacrifices valuable information and the latter introduces domain biases. Therefore, generating diverse-style source data without auxiliary data emerges as an attractive strategy. In light of this, we propose GAN-based feature augmentation (GBFA) that hallucinates stylized feature maps while preserving their semantic contents with a feature generator. The impressive generative capability of GANs enables GBFA to perform inter-channel and trainable feature synthesis in an end-to-end framework. To enable learning GBFA, we introduce random image color augmentation (RICA), which adds a diverse range of variations to source images during training. These augmented images are then passed through a feature extractor to obtain features tailored for GBFA training. Both GBFA and RICA operate exclusively within the source domain, eliminating the need for auxiliary datasets. We conduct extensive experiments, and the generalization results from the synthetic GTAV and SYNTHIA to the real Cityscapes, BDDS, and Mapillary datasets show that our method achieves state-of-the-art performance in DGSS.