Exploring Semantic Consistency in Unpaired Image Translation to Generate Data for Surgical Applications

TL;DR

This paper investigates unpaired image translation methods to generate realistic surgical images with preserved semantic structure, improving training data quality for surgical computer vision tasks.

Contribution

It provides an empirical evaluation of state-of-the-art models focusing on semantic consistency, highlighting the effectiveness of combining structural-similarity loss with contrastive learning.

Findings

Structural-similarity loss and contrastive learning improve semantic consistency.

Generated data enhances downstream segmentation performance.

Simple combined approach outperforms more complex models.

Abstract

In surgical computer vision applications, obtaining labeled training data is challenging due to data-privacy concerns and the need for expert annotation. Unpaired image-to-image translation techniques have been explored to automatically generate large annotated datasets by translating synthetic images to the realistic domain. However, preserving the structure and semantic consistency between the input and translated images presents significant challenges, mainly when there is a distributional mismatch in the semantic characteristics of the domains. This study empirically investigates unpaired image translation methods for generating suitable data in surgical applications, explicitly focusing on semantic consistency. We extensively evaluate various state-of-the-art image translation models on two challenging surgical datasets and downstream semantic segmentation tasks. We find that a simple combination of structural-similarity loss and contrastive learning yields the most promising results. Quantitatively, we show that the data generated with this approach yields higher semantic consistency and can be used more effectively as training data.The code is available at https://gitlab.com/nct_tso_public/constructs.