Enhancing Chest X-ray Classification through Knowledge Injection in Cross-Modality Learning

TL;DR

This paper introduces a novel knowledge injection framework that enhances chest X-ray classification accuracy by integrating fine-grained medical knowledge into a cross-modality learning model, demonstrating significant performance improvements.

Contribution

It proposes a Set Theory-based knowledge injection method that generates controllable medical captions, improving cross-modality classification performance in medical imaging.

Findings

Injecting fine-grained medical knowledge improves accuracy from 49.9% to 72.5%.

Using domain-specific LLMs for caption generation further enhances performance.

Knowledge density and specialized models positively impact classification results.

Abstract

The integration of artificial intelligence in medical imaging has shown tremendous potential, yet the relationship between pre-trained knowledge and performance in cross-modality learning remains unclear. This study investigates how explicitly injecting medical knowledge into the learning process affects the performance of cross-modality classification, focusing on Chest X-ray (CXR) images. We introduce a novel Set Theory-based knowledge injection framework that generates captions for CXR images with controllable knowledge granularity. Using this framework, we fine-tune CLIP model on captions with varying levels of medical information. We evaluate the model's performance through zero-shot classification on the CheXpert dataset, a benchmark for CXR classification. Our results demonstrate that injecting fine-grained medical knowledge substantially improves classification accuracy, achieving 72.5\% compared to 49.9\% when using human-generated captions. This highlights the crucial role of domain-specific knowledge in medical cross-modality learning. Furthermore, we explore the influence of knowledge density and the use of domain-specific Large Language Models (LLMs) for caption generation, finding that denser knowledge and specialized LLMs contribute to enhanced performance. This research advances medical image analysis by demonstrating the effectiveness of knowledge injection for improving automated CXR classification, paving the way for more accurate and reliable diagnostic tools.