Electrocardiogram-Language Model for Few-Shot Question Answering with Meta Learning

TL;DR

This paper presents a novel multimodal meta-learning approach that combines ECG signal encoding with large language models to enable few-shot question answering in clinical ECG interpretation, overcoming data scarcity and task diversity.

Contribution

It introduces a LLM-agnostic fusion method that integrates ECG encoders with frozen LLMs for improved few-shot ECG question answering performance.

Findings

Achieves 84.6% accuracy in 5-way 5-shot setting with LLaMA-3.1-8B.

Demonstrates superior generalization to unseen diagnostic tasks.

Effective with limited ECG leads and diverse question types.

Abstract

Electrocardiogram (ECG) interpretation requires specialized expertise, often involving synthesizing insights from ECG signals with complex clinical queries posed in natural language. The scarcity of labeled ECG data coupled with the diverse nature of clinical inquiries presents a significant challenge for developing robust and adaptable ECG diagnostic systems. This work introduces a novel multimodal meta-learning method for few-shot ECG question answering, addressing the challenge of limited labeled data while leveraging the rich knowledge encoded within large language models (LLMs). Our LLM-agnostic approach integrates a pre-trained ECG encoder with a frozen LLM (e.g., LLaMA and Gemma) via a trainable fusion module, enabling the language model to reason about ECG data and generate clinically meaningful answers. Extensive experiments demonstrate superior generalization to unseen diagnostic tasks compared to supervised baselines, achieving notable performance even with limited ECG leads. For instance, in a 5-way 5-shot setting, our method using LLaMA-3.1-8B achieves an accuracy of 84.6%, 77.3%, and 69.6% on single verify, choose and query question types, respectively. These results highlight the potential of our method to enhance clinical ECG interpretation by combining signal processing with the nuanced language understanding capabilities of LLMs, particularly in data-constrained scenarios.