Vispi: Automatic Visual Perception and Interpretation of Chest X-rays

TL;DR

Vispi is an automated system that classifies, localizes, and generates reports for chest X-rays, aiming to reduce radiologists' workload by improving medical image interpretation accuracy.

Contribution

It introduces a novel system combining disease classification, localization, and report generation tailored for chest X-rays, addressing data scarcity and model specificity issues.

Findings

Superior performance in disease classification and localization

Effective report generation validated by ROUGE and CIDEr metrics

Demonstrates potential to assist clinical diagnosis workflows

Abstract

Medical imaging contains the essential information for rendering diagnostic and treatment decisions. Inspecting (visual perception) and interpreting image to generate a report are tedious clinical routines for a radiologist where automation is expected to greatly reduce the workload. Despite rapid development of natural image captioning, computer-aided medical image visual perception and interpretation remain a challenging task, largely due to the lack of high-quality annotated image-report pairs and tailor-made generative models for sufficient extraction and exploitation of localized semantic features, particularly those associated with abnormalities. To tackle these challenges, we present Vispi, an automatic medical image interpretation system, which first annotates an image via classifying and localizing common thoracic diseases with visual support and then followed by report generation from an attentive LSTM model. Analyzing an open IU X-ray dataset, we demonstrate a superior performance of Vispi in disease classification, localization and report generation using automatic performance evaluation metrics ROUGE and CIDEr.