MedDChest: A Content-Aware Multimodal Foundational Vision Model for Thoracic Imaging

TL;DR

MedDChest is a specialized vision transformer model pre-trained on a large thoracic imaging dataset, utilizing a novel content-aware augmentation, which significantly improves performance on diagnostic tasks compared to models pre-trained on natural images.

Contribution

We introduce MedDChest, a domain-specific vision transformer trained from scratch on multimodal thoracic images, with a new content-aware data augmentation technique for improved medical imaging analysis.

Findings

MedDChest outperforms ImageNet-pretrained models on thoracic diagnostic tasks.

The content-aware augmentation enhances model focus on relevant anatomical regions.

Large-scale in-domain pre-training improves feature extraction for medical imaging.

Abstract

The performance of vision models in medical imaging is often hindered by the prevailing paradigm of fine-tuning backbones pre-trained on out-of-domain natural images. To address this fundamental domain gap, we propose MedDChest, a new foundational Vision Transformer (ViT) model optimized specifically for thoracic imaging. We pre-trained MedDChest from scratch on a massive, curated, multimodal dataset of over 1.2 million images, encompassing different modalities including Chest X-ray and Computed Tomography (CT) compiled from 10 public sources. A core technical contribution of our work is Guided Random Resized Crops, a novel content-aware data augmentation strategy that biases sampling towards anatomically relevant regions, overcoming the inefficiency of standard cropping techniques on medical scans. We validate our model's effectiveness by fine-tuning it on a diverse set of downstream diagnostic tasks. Comprehensive experiments empirically demonstrate that MedDChest significantly outperforms strong, publicly available ImageNet-pretrained models. By establishing the superiority of large-scale, in-domain pre-training combined with domain-specific data augmentation, MedDChest provides a powerful and robust feature extractor that serves as a significantly better starting point for a wide array of thoracic diagnostic tasks. The model weights will be made publicly available to foster future research and applications.