BAAI Cardiac Agent: An intelligent multimodal agent for automated reasoning and diagnosis of cardiovascular diseases from cardiac magnetic resonance imaging

TL;DR

BAAI Cardiac Agent is an advanced multimodal system that automates interpretation of cardiac MRI, achieving high accuracy and efficiency in diagnosing cardiovascular diseases by integrating expert models and generating clinical reports.

Contribution

The paper introduces BAAI Cardiac Agent, a novel end-to-end multimodal system that automates cardiac MRI analysis with high accuracy, outperforming existing models and streamlining clinical workflows.

Findings

Achieved AUC > 0.93 internally and > 0.81 externally in disease diagnosis.

Generated highly consistent cardiac function indices with clinical reports (Pearson > 0.90).

Outperformed state-of-the-art models in segmentation and diagnostic tasks.

Abstract

Cardiac magnetic resonance (CMR) is a cornerstone for diagnosing cardiovascular disease. However, it remains underutilized due to complex, time-consuming interpretation across multi-sequences, phases, quantitative measures that heavily reliant on specialized expertise. Here, we present BAAI Cardiac Agent, a multimodal intelligent system designed for end-to-end CMR interpretation. The agent integrates specialized cardiac expert models to perform automated segmentation of cardiac structures, functional quantification, tissue characterization and disease diagnosis, and generates structured clinical reports within a unified workflow. Evaluated on CMR datasets from two hospitals (2413 patients) spanning 7-types of major cardiovascular diseases, the agent achieved an area under the receiver-operating-characteristic curve exceeding 0.93 internally and 0.81 externally. In the task of estimating left ventricular function indices, the results generated by this system for core parameters such as ejection fraction, stroke volume, and left ventricular mass are highly consistent with clinical reports, with Pearson correlation coefficients all exceeding 0.90. The agent outperformed state-of-the-art models in segmentation and diagnostic tasks, and generated clinical reports showing high concordance with expert radiologists (six readers across three experience levels). By dynamically orchestrating expert models for coordinated multimodal analysis, this agent framework enables accurate, efficient CMR interpretation and highlights its potentials for complex clinical imaging workflows. Code is available at https://github.com/plantain-herb/Cardiac-Agent.