Dynamic Multi-Domain Knowledge Networks for Chest X-ray Report Generation

TL;DR

This paper introduces a Dynamic Multi-Domain Knowledge network that enhances chest X-ray report generation by addressing data imbalance and bias through dynamic knowledge extraction and integration, improving report accuracy and interpretability.

Contribution

The proposed DMDK network innovatively combines dynamic disease topic labels and knowledge graphs to mitigate visual and textual biases in radiology report generation.

Findings

Outperforms previous state-of-the-art models on IU X-Ray and MIMIC-CXR datasets.

Effectively highlights abnormal regions in X-ray images.

Improves interpretability of generated reports.

Abstract

The automated generation of radiology diagnostic reports helps radiologists make timely and accurate diagnostic decisions while also enhancing clinical diagnostic efficiency. However, the significant imbalance in the distribution of data between normal and abnormal samples (including visual and textual biases) poses significant challenges for a data-driven task like automatically generating diagnostic radiology reports. Therefore, we propose a Dynamic Multi-Domain Knowledge(DMDK) network for radiology diagnostic report generation. The DMDK network consists of four modules: Chest Feature Extractor(CFE), Dynamic Knowledge Extractor(DKE), Specific Knowledge Extractor(SKE), and Multi-knowledge Integrator(MKI) module. Specifically, the CFE module is primarily responsible for extracting the unprocessed visual medical features of the images. The DKE module is responsible for extracting dynamic disease topic labels from the retrieved radiology diagnostic reports. We then fuse the dynamic disease topic labels with the original visual features of the images to highlight the abnormal regions in the original visual features to alleviate the visual data bias problem. The SKE module expands upon the conventional static knowledge graph to mitigate textual data biases and amplify the interpretability capabilities of the model via domain-specific dynamic knowledge graphs. The MKI distills all the knowledge and generates the final diagnostic radiology report. We performed extensive experiments on two widely used datasets, IU X-Ray and MIMIC-CXR. The experimental results demonstrate the effectiveness of our method, with all evaluation metrics outperforming previous state-of-the-art models.