TL;DR
HeartcareGPT introduces a unified multimodal ECG model with a new dataset, benchmark, and dual encoder alignment, advancing cross-modal ECG understanding and modeling.
Contribution
The paper presents HeartcareGPT, a novel multimodal ECG model with a dedicated dataset, benchmark, and a dual encoder alignment paradigm for improved ECG analysis.
Findings
HeartcareGPT achieves consistent improvements across ECG understanding tasks.
The dataset Heartcare-400K enhances model training with high-quality clinical reports.
The dual encoder projection alignment improves joint signal-image modeling.
Abstract
Although electrocardiograms (ECG) play a dominant role in cardiovascular diagnosis and treatment, their intrinsic data forms and representational patterns pose significant challenges for medical multimodal large language models (Med-MLLMs) in achieving cross-modal semantic alignment. To address this gap, we propose Heartcare Suite, a unified ECG suite designed for dual signal-image modeling and understanding: (i) Heartcare-400K. A fine-grained ECG instruction dataset on top of our data pipeline engine--HeartAgent--by integrating high quality clinical ECG reports from top hospitals with open-source data. (ii) Heartcare-Bench. A systematic benchmark assessing performance of models in multi-perspective ECG understanding and cross-modal generalization, providing guidance for optimizing ECG comprehension models. (iii) HeartcareGPT. Built upon a structure-aware discrete tokenizer Beat, we…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
