# ViT-Stain: Vision transformer-driven virtual staining for skin histopathology via global contextual learning

## TL;DR

ViT-Stain uses vision transformers to create high-quality virtual H&E-stained skin tissue images, outperforming traditional methods in preserving detail and accuracy.

## Contribution

Introduces ViT-Stain, a vision transformer-based framework for virtual staining that captures global context and improves diagnostic fidelity.

## Key findings

- ViT-Stain outperforms CNN and GAN models in virtual staining metrics like SSIM, PSNR, and FID.
- The model achieves 85% diagnostic concordance with H&E-stained images, as evaluated by pathologists.
- A novel histology-specific fidelity index (HSFI) was developed and used for evaluation.

## Abstract

Current virtual staining approaches for histopathology slides use convolutional neural networks (CNNs) and generative adversarial networks (GANs). These approaches rely on local receptive fields, struggle to capture global context, and long-range tissue dependencies. This limitation can introduce artifacts in fine textures and cause loss of subtle morphological details. We propose a novel vision transformer-driven virtual staining framework (ViT-Stain) that translates unstained skin tissue images into hematoxylin and eosin (H&E)-equivalent images. The transformer’s self-attention enables ViT-Stain to capture long-range dependencies, preserve global context, and maintain fine textures. We trained ViT-Stain on the E-Staining DermaRepo dataset, which pairs unstained and H&E-stained whole-slide images (WSIs). We validated our model using metrics including SSIM, PSNR, FID, KID, LPIPS, and a novel histology-specific fidelity index (HSFI). Three board-certified pathologists provided feedback for qualitative evaluations. ViT-Stain outperforms leading CNN and GAN models, including Pix2Pix, CycleGAN, CUTGAN, and DCLGAN. It achieves an overall diagnostic concordance of 85% with virtual H&E-stains (Fleiss’ κ = 0.88). However, the model requires longer training (about 93 hours on A100 GPUs) and inference times (about 2.9 minutes). Our work advances AI-driven diagnostic reproducibility for high-fidelity clinical settings and aligns with the World Health Organization (WHO) global health goals.