Adaptive H&E-IHC information fusion staining framework based on feature extra

TL;DR

This paper introduces an adaptive feature-based framework for H&E-IHC image fusion that enhances color information extraction and improves staining quality by using contrastive learning and wavelet transforms.

Contribution

It proposes a novel adaptive information enhancement framework with a dual feature extractor trained via contrastive learning for better H&E-IHC image fusion.

Findings

Effective color feature extraction with wavelet transform convolution.

Improved H&E-IHC image fusion performance on multiple datasets.

Enhanced feature alignment using contrastive learning.

Abstract

Immunohistochemistry (IHC) staining plays a significant role in the evaluation of diseases such as breast cancer. The H&E-to-IHC transformation based on generative models provides a simple and cost-effective method for obtaining IHC images. Although previous models can perform digital coloring well, they still suffer from (i) coloring only through the pixel features that are not prominent in HE, which is easy to cause information loss in the coloring process; (ii) The lack of pixel-perfect H&E-IHC groundtruth pairs poses a challenge to the classical L1 loss.To address the above challenges, we propose an adaptive information enhanced coloring framework based on feature extractors. We first propose the VMFE module to effectively extract the color information features using multi-scale feature extraction and wavelet transform convolution, while combining the shared decoder for feature fusion. The high-performance dual feature extractor of H&E-IHC is trained by contrastive learning, which can effectively perform feature alignment of HE-IHC in high latitude space. At the same time, the trained feature encoder is used to enhance the features and adaptively adjust the loss in the HE section staining process to solve the problems related to unclear and asymmetric information. We have tested on different datasets and achieved excellent performance.Our code is available at https://github.com/babyinsunshine/CEFF