A versatile foundation model for cine cardiac magnetic resonance image analysis tasks

TL;DR

This paper introduces CineMA, a versatile foundation model trained on 15 million cine CMR images, capable of performing multiple cardiac image analysis tasks with high accuracy, efficiency, and fairness, outperforming traditional CNNs and applicable across diverse populations.

Contribution

CineMA is the first multi-task, multi-view transformer-based foundation model for cine cardiac MRI analysis, demonstrating superior performance and generalizability over existing CNN models.

Findings

CineMA outperforms CNNs in ventricular segmentation and ejection fraction estimation.

CineMA maintains high performance with limited fine-tuning data.

CineMA can detect systemic disease-related cardiac changes and predict mortality.

Abstract

Here we present a versatile foundation model that can perform a range of clinically-relevant image analysis tasks, including segmentation, landmark localisation, diagnosis, and prognostication. A multi-view convolution-transformer masked autoencoder, named as CineMA, was trained on 15 million cine images from 74,916 subjects. The model was validated on multiple image analysis tasks and compared to existing models on >4,500 images from eight independent datasets with diverse population characteristics, representing the largest benchmark study for cine CMR so far. CineMA consistently outperformed conventional convolutional neural networks (CNNs) in delineating ventricular boundaries and estimating ejection fraction, a key measure of cardiac function. The improved performance was preserved, even when the model only used half of fine-tuning data. CineMA also surpassed CNNs in disease detection and matched their performance in long-axis function measurement. Interestingly, we found that CineMA can also detect cardiac changes in systemic diseases, such as diabetes, hypertension and cancer, and can also predict mortality. Finally, we assessed model fairness and demonstrated consistent model performance across demographic subgroups. These findings highlight CineMA's accuracy, learning efficiency, adaptability, and fairness, underscoring its potential as a foundation model for automated cardiac image analysis to support clinical workflow and cardiovascular research. All training and inference code and models are made publicly available at https://github.com/mathpluscode/CineMA.