Anatomically Constrained Transformers for Echocardiogram Analysis

TL;DR

This paper introduces ViACT, a transformer framework for echocardiogram analysis that incorporates anatomical priors to improve focus on diagnostic regions, leading to more interpretable and task-specific models.

Contribution

ViACT uniquely integrates anatomical priors into transformer architecture, enabling focused learning on cardiac regions and improving interpretability in echo analysis.

Findings

ViACT improves focus on myocardium, enhancing interpretability.

Pre-training with anatomical masking enhances task-specific performance.

ViACT generalizes to myocardium tracking without specialized components.

Abstract

Video transformers have recently demonstrated strong potential for echocardiogram (echo) analysis, leveraging self-supervised pre-training and flexible adaptation across diverse tasks. However, like other models operating on videos, they are prone to learning spurious correlations from non-diagnostic regions such as image backgrounds. To overcome this limitation, we propose the Video Anatomically Constrained Transformer (ViACT), a novel framework that integrates anatomical priors directly into the transformer architecture. ViACT represents a deforming anatomical structure as a point set and encodes both its spatial geometry and corresponding image patches into transformer tokens. During pre-training, ViACT follows a masked autoencoding strategy that masks and reconstructs only anatomical patches, enforcing that representation learning is focused on the anatomical region. The pre-trained model can then be fine-tuned for tasks localized to this region. In this work we focus on the myocardium, demonstrating the framework on echo analysis tasks such as left ventricular ejection fraction (EF) regression and cardiac amyloidosis (CA) detection. The anatomical constraint focuses transformer attention within the myocardium, yielding interpretable attention maps aligned with regions of known CA pathology. Moreover, ViACT generalizes to myocardium point tracking without requiring task-specific components such as correlation volumes used in specialized tracking networks.