USIGAN: Unbalanced Self-Information Feature Transport for Weakly Paired Image IHC Virtual Staining

TL;DR

USIGAN introduces a novel method for virtual IHC staining that effectively handles weakly paired data, improving content and semantic consistency in generated images through unbalanced feature transport and specialized mechanisms.

Contribution

The paper proposes USIGAN, a new unbalanced self-information feature transport approach with mechanisms to improve weakly paired IHC virtual staining accuracy.

Findings

Achieves superior performance on clinical metrics like IoD and Pearson-R correlation.

Effectively mitigates weak pairing issues, enhancing content and semantic consistency.

Demonstrates improved results on two public datasets.

Abstract

Immunohistochemical (IHC) virtual staining is a task that generates virtual IHC images from H\&E images while maintaining pathological semantic consistency with adjacent slices. This task aims to achieve cross-domain mapping between morphological structures and staining patterns through generative models, providing an efficient and cost-effective solution for pathological analysis. However, under weakly paired conditions, spatial heterogeneity between adjacent slices presents significant challenges. This can lead to inaccurate one-to-many mappings and generate results that are inconsistent with the pathological semantics of adjacent slices. To address this issue, we propose a novel unbalanced self-information feature transport for IHC virtual staining, named USIGAN, which extracts global morphological semantics without relying on positional correspondence.By removing weakly paired terms in the joint marginal distribution, we effectively mitigate the impact of weak pairing on joint distributions, thereby significantly improving the content consistency and pathological semantic consistency of the generated results. Moreover, we design the Unbalanced Optimal Transport Consistency (UOT-CTM) mechanism and the Pathology Self-Correspondence (PC-SCM) mechanism to construct correlation matrices between H\&E and generated IHC in image-level and real IHC and generated IHC image sets in intra-group level.. Experiments conducted on two publicly available datasets demonstrate that our method achieves superior performance across multiple clinically significant metrics, such as IoD and Pearson-R correlation, demonstrating better clinical relevance.