Chest X-ray Inpainting with Deep Generative Models

TL;DR

This paper evaluates deep generative models for chest X-ray inpainting, demonstrating their realistic outputs and potential for abnormality detection, with a focus on model performance and human observer challenges.

Contribution

It compares three recent deep learning inpainting models on chest X-rays, highlighting their realism and potential clinical utility in abnormality detection.

Findings

Models produce highly realistic inpainted regions.

The models show potential for abnormality detection.

Humans struggle to identify inpainted regions, especially with the contextual attention model.

Abstract

Generative adversarial networks have been successfully applied to inpainting in natural images. However, the current state-of-the-art models have not yet been widely adopted in the medical imaging domain. In this paper, we investigate the performance of three recently published deep learning based inpainting models: context encoders, semantic image inpainting, and the contextual attention model, applied to chest x-rays, as the chest exam is the most commonly performed radiological procedure. We train these generative models on 1.2M 128 $\times$ 128 patches from 60K healthy x-rays, and learn to predict the center 64 $\times$ 64 region in each patch. We test the models on both the healthy and abnormal radiographs. We evaluate the results by visual inspection and comparing the PSNR scores. The outputs of the models are in most cases highly realistic. We show that the methods have potential to enhance and detect abnormalities. In addition, we perform a 2AFC observer study and show that an experienced human observer performs poorly in detecting inpainted regions, particularly those generated by the contextual attention model.