Slide-free MUSE Microscopy to H&E Histology Modality Conversion via Unpaired Image-to-Image Translation GAN Models

TL;DR

This paper explores converting MUSE slide-free tissue images into traditional H&E-stained images using unpaired image-to-image translation GAN models, aiming to facilitate adoption of new imaging techniques.

Contribution

It evaluates multiple GAN models for MUSE to H&E conversion, identifying CycleGAN as the most effective and highlighting the importance of MUSE color inversion.

Findings

CycleGAN achieved the best visual and performance results.

MUSE color inversion improves modality conversion accuracy.

CycleGAN outperforms other models in style transfer and content preservation.

Abstract

MUSE is a novel slide-free imaging technique for histological examination of tissues that can serve as an alternative to traditional histology. In order to bridge the gap between MUSE and traditional histology, we aim to convert MUSE images to resemble authentic hematoxylin- and eosin-stained (H&E) images. We evaluated four models: a non-machine-learning-based color-mapping unmixing-based tool, CycleGAN, DualGAN, and GANILLA. CycleGAN and GANILLA provided visually compelling results that appropriately transferred H&E style and preserved MUSE content. Based on training an automated critic on real and generated H&E images, we determined that CycleGAN demonstrated the best performance. We have also found that MUSE color inversion may be a necessary step for accurate modality conversion to H&E. We believe that our MUSE-to-H&E model can help improve adoption of novel slide-free methods by bridging a perceptual gap between MUSE imaging and traditional histology.