Building Trust in Virtual Immunohistochemistry: Automated Assessment of Image Quality

TL;DR

This paper presents an automated, accuracy-based framework for evaluating virtual immunohistochemistry image quality, highlighting the limitations of traditional fidelity metrics and emphasizing the importance of stain accuracy in clinical applications.

Contribution

Introduces a novel, automated method to assess virtual IHC image quality based on stain accuracy metrics, improving reliability over conventional fidelity measures.

Findings

Conventional metrics like FID, PSNR, SSIM poorly correlate with stain accuracy.

Paired models like PyramidPix2Pix achieve higher stain accuracy.

Whole-slide image evaluation reveals performance issues not seen in patch-based analysis.

Abstract

Deep learning models can generate virtual immunohistochemistry (IHC) stains from hematoxylin and eosin (H&E) images, offering a scalable and low-cost alternative to laboratory IHC. However, reliable evaluation of image quality remains a challenge as current texture- and distribution-based metrics quantify image fidelity rather than the accuracy of IHC staining. Here, we introduce an automated and accuracy grounded framework to determine image quality across sixteen paired or unpaired image translation models. Using color deconvolution, we generate masks of pixels stained brown (i.e., IHC-positive) as predicted by each virtual IHC model. We use the segmented masks of real and virtual IHC to compute stain accuracy metrics (Dice, IoU, Hausdorff distance) that directly quantify correct pixel - level labeling without needing expert manual annotations. Our results demonstrate that conventional image fidelity metrics, including Frechet Inception Distance (FID), peak signal-to-noise ratio (PSNR), and structural similarity (SSIM), correlate poorly with stain accuracy and pathologist assessment. Paired models such as PyramidPix2Pix and AdaptiveNCE achieve the highest stain accuracy, whereas unpaired diffusion- and GAN-based models are less reliable in providing accurate IHC positive pixel labels. Moreover, whole-slide images (WSI) reveal performance declines that are invisible in patch-based evaluations, emphasizing the need for WSI-level benchmarks. Together, this framework defines a reproducible approach for assessing the quality of virtual IHC models, a critical step to accelerate translation towards routine use by pathologists.