Enhanced Unsupervised Image-to-Image Translation Using Contrastive Learning and Histogram of Oriented Gradients

TL;DR

This paper introduces an improved unsupervised image-to-image translation method that combines contrastive learning with HOG features to better preserve image structure and reduce artifacts, especially in unpaired data scenarios.

Contribution

It integrates HOG features into the CUT model to enhance semantic preservation without requiring labels, improving image quality and reducing hallucinations.

Findings

Significant reduction in image artifacts and hallucinations.

Improved semantic structure preservation in translated images.

Enhanced image quality in synthetic to real scene translation.

Abstract

Image-to-Image Translation is a vital area of computer vision that focuses on transforming images from one visual domain to another while preserving their core content and structure. However, this field faces two major challenges: first, the data from the two domains are often unpaired, making it difficult to train generative adversarial networks effectively; second, existing methods tend to produce artifacts or hallucinations during image generation, leading to a decline in image quality. To address these issues, this paper proposes an enhanced unsupervised image-to-image translation method based on the Contrastive Unpaired Translation (CUT) model, incorporating Histogram of Oriented Gradients (HOG) features. This novel approach ensures the preservation of the semantic structure of images, even without semantic labels, by minimizing the loss between the HOG features of input and generated images. The method was tested on translating synthetic game environments from GTA5 dataset to realistic urban scenes in cityscapes dataset, demonstrating significant improvements in reducing hallucinations and enhancing image quality.