Content Preserving Image Translation with Texture Co-occurrence and Spatial Self-Similarity for Texture Debiasing and Domain Adaptation

TL;DR

This paper introduces a novel image translation framework that reduces texture bias in models by generating augmented training data with preserved content and controlled texture, improving robustness in classification and segmentation tasks.

Contribution

The proposed method explicitly mitigates texture bias by combining texture co-occurrence and spatial self-similarity losses during image translation, outperforming recent state-of-the-art approaches.

Findings

Significant performance improvements on five classification datasets.

Enhanced segmentation accuracy on two datasets with texture bias.

Effective reduction of texture bias in trained models.

Abstract

Models trained on datasets with texture bias usually perform poorly on out-of-distribution samples since biased representations are embedded into the model. Recently, various image translation and debiasing methods have attempted to disentangle texture biased representations for downstream tasks, but accurately discarding biased features without altering other relevant information is still challenging. In this paper, we propose a novel framework that leverages image translation to generate additional training images using the content of a source image and the texture of a target image with a different bias property to explicitly mitigate texture bias when training a model on a target task. Our model ensures texture similarity between the target and generated images via a texture co-occurrence loss while preserving content details from source images with a spatial self-similarity loss. Both the generated and original training images are combined to train improved classification or segmentation models robust to inconsistent texture bias. Evaluation on five classification- and two segmentation-datasets with known texture biases demonstrates the utility of our method, and reports significant improvements over recent state-of-the-art methods in all cases.