Restoration of marker occluded hematoxylin and eosin stained whole slide histology images using generative adversarial networks

TL;DR

This paper presents a generative adversarial network approach to remove marker annotations from histology images, restoring tissue details for digital pathology analysis with high accuracy and fidelity.

Contribution

It introduces a cycle-consistent GAN method for marker removal in whole slide images, achieving near-human indistinguishability and high correlation with original tissue structures.

Findings

70% of corrected patches indistinguishable from original tissue by humans

97% indistinguishable when evaluated by a deep residual network

Restoration of up to 94,000 nuclei per slide, mainly on tissue borders

Abstract

It is common for pathologists to annotate specific regions of the tissue, such as tumor, directly on the glass slide with markers. Although this practice was helpful prior to the advent of histology whole slide digitization, it often occludes important details which are increasingly relevant to immuno-oncology due to recent advancements in digital pathology imaging techniques. The current work uses a generative adversarial network with cycle loss to remove these annotations while still maintaining the underlying structure of the tissue by solving an image-to-image translation problem. We train our network on up to 300 whole slide images with marker inks and show that 70% of the corrected image patches are indistinguishable from originally uncontaminated image tissue to a human expert. This portion increases 97% when we replace the human expert with a deep residual network. We demonstrated the fidelity of the method to the original image by calculating the correlation between image gradient magnitudes. We observed a revival of up to 94,000 nuclei per slide in our dataset, the majority of which were located on tissue border.