Clinically-aligned Multi-modal Chest X-ray Classification

TL;DR

This paper presents CaMCheX, a multimodal transformer framework that integrates multi-view chest X-ray images with clinical data to improve diagnostic classification, surpassing existing methods on benchmark datasets.

Contribution

Introduces CaMCheX, a novel multimodal transformer architecture that aligns multi-view X-ray images with structured clinical data for enhanced classification accuracy.

Findings

Outperforms state-of-the-art on MIMIC-CXR and CXR-LT datasets.

Demonstrates the effectiveness of multimodal clinical alignment.

Highlights the importance of multi-view and clinical data integration.

Abstract

Radiology is essential to modern healthcare, yet rising demand and staffing shortages continue to pose major challenges. Recent advances in artificial intelligence have the potential to support radiologists and help address these challenges. Given its widespread use and clinical importance, chest X-ray classification is well suited to augment radiologists' workflows. However, most existing approaches rely solely on single-view, image-level inputs, ignoring the structured clinical information and multi-image studies available at the time of reporting. In this work, we introduce CaMCheX, a multimodal transformer-based framework that aligns multi-view chest X-ray studies with structured clinical data to better reflect how clinicians make diagnostic decisions. Our architecture employs view-specific ConvNeXt encoders for frontal and lateral chest radiographs, whose features are fused with clinical indications, history, and vital signs using a transformer fusion module. This design enables the model to generate context-aware representations that mirror reasoning in clinical practice. Our results exceed the state of the art for both the original MIMIC-CXR dataset and the more recent CXR-LT benchmarks, highlighting the value of clinically grounded multimodal alignment for advancing chest X-ray classification.