Unpaired Image-to-Image Translation for Segmentation and Signal Unmixing

TL;DR

Ui2i is a new unpaired image-to-image translation model that enhances content preservation and disentanglement, enabling accurate biomedical image translation and signal unmixing without paired datasets.

Contribution

Ui2i introduces modifications to CycleGAN, including U-Net generators, spectral normalization, and attention mechanisms, to improve content fidelity and enable signal separation in unpaired biomedical images.

Findings

Successfully applied to nuclear segmentation in IHC images.

Achieved unmixing of superimposed signals in IF images.

Demonstrated improved content preservation over existing methods.

Abstract

This work introduces Ui2i, a novel model for unpaired image-to-image translation, trained on content-wise unpaired datasets to enable style transfer across domains while preserving content. Building on CycleGAN, Ui2i incorporates key modifications to better disentangle content and style features, and preserve content integrity. Specifically, Ui2i employs U-Net-based generators with skip connections to propagate localized shallow features deep into the generator. Ui2i removes feature-based normalization layers from all modules and replaces them with approximate bidirectional spectral normalization -- a parameter-based alternative that enhances training stability. To further support content preservation, channel and spatial attention mechanisms are integrated into the generators. Training is facilitated through image scale augmentation. Evaluation on two biomedical tasks -- domain adaptation for nuclear segmentation in immunohistochemistry (IHC) images and unmixing of biological structures superimposed in single-channel immunofluorescence (IF) images -- demonstrates Ui2i's ability to preserve content fidelity in settings that demand more accurate structural preservation than typical translation tasks. To the best of our knowledge, Ui2i is the first approach capable of separating superimposed signals in IF images using real, unpaired training data.