Cyclic Generative Adversarial Networks With Congruent Image-Report Generation For Explainable Medical Image Analysis

TL;DR

This paper introduces a cyclic GAN framework that generates congruent medical image-report pairs, enhancing explainability and trustworthiness in chest X-ray diagnosis by providing faithful visual and textual explanations.

Contribution

The novel cyclic GAN approach jointly generates consistent images and reports, improving interpretability and achieving state-of-the-art results in medical image labeling.

Findings

Achieved state-of-the-art performance on Indiana Chest X-ray dataset

Generated congruent image-report pairs for explainability

Enhanced trustworthiness of medical image diagnosis

Abstract

We present a novel framework for explainable labeling and interpretation of medical images. Medical images require specialized professionals for interpretation, and are explained (typically) via elaborate textual reports. Different from prior methods that focus on medical report generation from images or vice-versa, we novelly generate congruent image--report pairs employing a cyclic-Generative Adversarial Network (cycleGAN); thereby, the generated report will adequately explain a medical image, while a report-generated image that effectively characterizes the text visually should (sufficiently) resemble the original. The aim of the work is to generate trustworthy and faithful explanations for the outputs of a model diagnosing chest x-ray images by pointing a human user to similar cases in support of a diagnostic decision. Apart from enabling transparent medical image labeling and interpretation, we achieve report and image-based labeling comparable to prior methods, including state-of-the-art performance in some cases as evidenced by experiments on the Indiana Chest X-ray dataset