Texture Underfitting for Domain Adaptation

TL;DR

This paper introduces a training method that reduces texture overfitting in neural networks to enhance domain adaptation for semantic segmentation, especially from synthetic to real images, by employing random image stylization.

Contribution

It proposes a novel training procedure that encourages texture underfitting, improving domain adaptation performance in semantic segmentation tasks.

Findings

Outperforms conventional training methods in synthetic-to-real domain adaptation.

Effective in both supervised and unsupervised segmentation scenarios.

Enhances generalization by reducing texture bias in neural networks.

Abstract

Comprehensive semantic segmentation is one of the key components for robust scene understanding and a requirement to enable autonomous driving. Driven by large scale datasets, convolutional neural networks show impressive results on this task. However, a segmentation algorithm generalizing to various scenes and conditions would require an enormously diverse dataset, making the labour intensive data acquisition and labeling process prohibitively expensive. Under the assumption of structural similarities between segmentation maps, domain adaptation promises to resolve this challenge by transferring knowledge from existing, potentially simulated datasets to new environments where no supervision exists. While the performance of this approach is contingent on the concept that neural networks learn a high level understanding of scene structure, recent work suggests that neural networks are biased towards overfitting to texture instead of learning structural and shape information. Considering the ideas underlying semantic segmentation, we employ random image stylization to augment the training dataset and propose a training procedure that facilitates texture underfitting to improve the performance of domain adaptation. In experiments with supervised as well as unsupervised methods for the task of synthetic-to-real domain adaptation, we show that our approach outperforms conventional training methods.