TPA: Temporal Prompt Alignment for Fetal Congenital Heart Defect Classification

TL;DR

This paper introduces TPA, a novel framework that leverages temporal modeling, prompt-aware contrastive learning, and uncertainty quantification to improve fetal CHD classification accuracy and calibration in ultrasound videos.

Contribution

The paper proposes TPA, combining foundation image-text models with temporal feature extraction and a new calibration module, advancing fetal CHD detection methods.

Findings

Achieves 85.40% macro F1 on private CHD dataset.

Reduces calibration error by 5.38%.

Improves F1 score by 4.73% on EchoNet-Dynamic.

Abstract

Congenital heart defect (CHD) detection in ultrasound videos is hindered by image noise and probe positioning variability. While automated methods can reduce operator dependence, current machine learning approaches often neglect temporal information, limit themselves to binary classification, and do not account for prediction calibration. We propose Temporal Prompt Alignment (TPA), a method leveraging foundation image-text model and prompt-aware contrastive learning to classify fetal CHD on cardiac ultrasound videos. TPA extracts features from each frame of video subclips using an image encoder, aggregates them with a trainable temporal extractor to capture heart motion, and aligns the video representation with class-specific text prompts via a margin-hinge contrastive loss. To enhance calibration for clinical reliability, we introduce a Conditional Variational Autoencoder Style Modulation (CVAESM) module, which learns a latent style vector to modulate embeddings and quantifies classification uncertainty. Evaluated on a private dataset for CHD detection and on a large public dataset, EchoNet-Dynamic, for systolic dysfunction, TPA achieves state-of-the-art macro F1 scores of 85.40% for CHD diagnosis, while also reducing expected calibration error by 5.38% and adaptive ECE by 6.8%. On EchoNet-Dynamic's three-class task, it boosts macro F1 by 4.73% (from 53.89% to 58.62%). Temporal Prompt Alignment (TPA) is a framework for fetal congenital heart defect (CHD) classification in ultrasound videos that integrates temporal modeling, prompt-aware contrastive learning, and uncertainty quantification.