From Foundation ECG Models to NISQ Learners: Distilling ECGFounder into a VQC Student

TL;DR

This paper explores distilling a large ECG foundation model into smaller classical and quantum-ready models, achieving competitive accuracy with fewer parameters and analyzing performance trade-offs.

Contribution

It introduces a method to transfer knowledge from a foundation ECG model to compact classical and quantum-ready learners, including a novel quantum pipeline.

Findings

Distillation improves performance of small models while reducing parameters.

Quantum-ready models achieve competitive accuracy with classical counterparts.

Performance sensitivity depends on distillation settings and model complexity.

Abstract

Foundation models have recently improved electrocardiogram (ECG) representation learning, but their deployment can be limited by computational cost and latency constraints. In this work, we fine-tune ECGFounder as a high-capacity teacher for binary ECG classification on PTB-XL and the MIT-BIH Arrhythmia Database, and investigate whether knowledge distillation can transfer its predictive behavior to compact students. We evaluate two classical 1D students (ResNet-1D and a lightweight CNN-1D) and a quantum-ready pipeline that combines a convolutional autoencoder, which compresses 256-sample ECG windows into a low-dimensional latent representation, with a 6-qubit variational quantum circuit implemented in Qiskit and executed in a simulated backend. Across both datasets, the teacher provides the strongest overall performance, while distillation yields competitive students under a considerable reduction in trainable parameters. We further analyze the sensitivity of student performance to distillation settings, highlighting consistent accuracy--efficiency trade-offs when compressing a foundation ECG model into classical and quantum-ready learners under a unified evaluation protocol.