ECG-Agent: On-Device Tool-Calling Agent for ECG Multi-Turn Dialogue

TL;DR

ECG-Agent is a novel on-device multi-turn ECG dialogue system that improves response accuracy and tool-calling ability, enabling real-world ECG analysis with efficient large language models.

Contribution

This paper introduces ECG-Agent, the first LLM-based tool-calling agent for multi-turn ECG dialogue, along with a new dataset for training and evaluation.

Findings

ECG-Agents outperform baseline models in response accuracy.

On-device ECG-Agents achieve comparable performance to larger models.

ECG-Agents demonstrate effective tool-calling and reduced hallucinations.

Abstract

Recent advances in Multimodal Large Language Models have rapidly expanded to electrocardiograms, focusing on classification, report generation, and single-turn QA tasks. However, these models fall short in real-world scenarios, lacking multi-turn conversational ability, on-device efficiency, and precise understanding of ECG measurements such as the PQRST intervals. To address these limitations, we introduce ECG-Agent, the first LLM-based tool-calling agent for multi-turn ECG dialogue. To facilitate its development and evaluation, we also present ECG-Multi-Turn-Dialogue (ECG-MTD) dataset, a collection of realistic user-assistant multi-turn dialogues for diverse ECG lead configurations. We develop ECG-Agents in various sizes, from on-device capable to larger agents. Experimental results show that ECG-Agents outperform baseline ECG-LLMs in response accuracy. Furthermore, on-device agents achieve comparable performance to larger agents in various evaluations that assess response accuracy, tool-calling ability, and hallucinations, demonstrating their viability for real-world applications.