Leveraging Pre-trained Models for FF-to-FFPE Histopathological Image Translation

TL;DR

This paper introduces Diffusion-FFPE, a novel method that uses a pre-trained diffusion model with multi-scale feature fusion and a vision-language discriminator to translate FF histopathological images into FFPE style, improving image quality for diagnosis.

Contribution

The paper presents a new FF-to-FFPE image translation approach leveraging a fine-tuned diffusion model, multi-scale feature fusion, and a vision-language discriminator, outperforming existing methods.

Findings

Outperforms existing FF-to-FFPE translation methods on TCGA-NSCLC dataset.

Effectively captures both global and local image features.

Demonstrates high-quality image translation suitable for diagnostic use.

Abstract

The two primary types of Hematoxylin and Eosin (H&E) slides in histopathology are Formalin-Fixed Paraffin-Embedded (FFPE) and Fresh Frozen (FF). FFPE slides offer high quality histopathological images but require a labor-intensive acquisition process. In contrast, FF slides can be prepared quickly, but the image quality is relatively poor. Our task is to translate FF images into FFPE style, thereby improving the image quality for diagnostic purposes. In this paper, we propose Diffusion-FFPE, a method for FF-to-FFPE histopathological image translation using a pre-trained diffusion model. Specifically, we utilize a one-step diffusion model as the generator, which we fine-tune using LoRA adapters within an adversarial learning framework. To enable the model to effectively capture both global structural patterns and local details, we introduce a multi-scale feature fusion module that leverages two VAE encoders to extract features at different image resolutions, performing feature fusion before inputting them into the UNet. Additionally, a pre-trained vision-language model for histopathology serves as the backbone for the discriminator, enhancing model performance. Our FF-to-FFPE translation experiments on the TCGA-NSCLC dataset demonstrate that the proposed approach outperforms existing methods. The code and models are released at https://github.com/QilaiZhang/Diffusion-FFPE.