RobustNet: Improving Domain Generalization in Urban-Scene Segmentation via Instance Selective Whitening

TL;DR

RobustNet introduces an instance selective whitening loss that enhances the domain generalization of urban-scene segmentation networks, enabling better performance on unseen and challenging conditions without extra computational cost.

Contribution

The paper proposes a novel instance selective whitening loss that disentangles style and content in features, improving robustness to domain shifts in urban-scene segmentation.

Findings

Significant performance improvements on unseen domains.

Effective across various backbone networks.

No additional computational cost.

Abstract

Enhancing the generalization capability of deep neural networks to unseen domains is crucial for safety-critical applications in the real world such as autonomous driving. To address this issue, this paper proposes a novel instance selective whitening loss to improve the robustness of the segmentation networks for unseen domains. Our approach disentangles the domain-specific style and domain-invariant content encoded in higher-order statistics (i.e., feature covariance) of the feature representations and selectively removes only the style information causing domain shift. As shown in Fig. 1, our method provides reasonable predictions for (a) low-illuminated, (b) rainy, and (c) unseen structures. These types of images are not included in the training dataset, where the baseline shows a significant performance drop, contrary to ours. Being simple yet effective, our approach improves the robustness of various backbone networks without additional computational cost. We conduct extensive experiments in urban-scene segmentation and show the superiority of our approach to existing work. Our code is available at https://github.com/shachoi/RobustNet.