Q-Heart: ECG Question Answering via Knowledge-Informed Multimodal LLMs

TL;DR

Q-Heart is a multimodal framework that combines ECG signals and clinical text to answer complex medical questions, achieving state-of-the-art accuracy in ECG question answering tasks.

Contribution

It introduces a novel ECG-aware transformer architecture with knowledge-informed prompting and retrieval, enhancing multimodal reasoning for ECG-based clinical question answering.

Findings

Achieves 4% higher exact match accuracy than previous methods.

Effectively integrates ECG signals with clinical reports for improved reasoning.

Demonstrates potential for more interpretable and capable clinical decision support systems.

Abstract

Electrocardiography (ECG) offers critical cardiovascular insights, such as identifying arrhythmias and myocardial ischemia, but enabling automated systems to answer complex clinical questions directly from ECG signals (ECG-QA) remains a significant challenge. Current approaches often lack robust multimodal reasoning capabilities or rely on generic architectures ill-suited for the nuances of physiological signals. We introduce Q-Heart, a novel multimodal framework designed to bridge this gap. Q-Heart leverages a powerful, adapted ECG encoder and integrates its representations with textual information via a specialized ECG-aware transformer-based mapping layer. Furthermore, Q-Heart leverages dynamic prompting and retrieval of relevant historical clinical reports to guide tuning the language model toward knowledge-aware ECG reasoning. Extensive evaluations on the benchmark ECG-QA dataset show Q-Heart achieves state-of-the-art performance, outperforming existing methods by a 4% improvement in exact match accuracy. Our work demonstrates the effectiveness of combining domain-specific architectural adaptations with knowledge-augmented LLM instruction tuning for complex physiological ECG analysis, paving the way for more capable and potentially interpretable clinical patient care systems.