Beyond Independent Frames: Latent Attention Masked Autoencoders for Multi-View Echocardiography

TL;DR

This paper introduces LAMAE, a multi-view masked autoencoder for echocardiography that captures cross-view information in latent space, improving cardiac representation and transferability across patient groups.

Contribution

LAMAE is the first model to incorporate multi-view latent attention in masked autoencoders for echocardiography, enabling holistic cardiac analysis from partial, multi-view data.

Findings

LAMAE effectively reconstructs cardiac function from partial multi-view observations.

Pretraining on MIMIC-IV-ECHO improves ICD-10 code prediction accuracy.

Transfer learning from adult to pediatric data remains effective despite anatomical differences.

Abstract

Echocardiography is a widely used modality for cardiac assessment due to its non-invasive and cost-effective nature, but the sparse and heterogeneous spatiotemporal views of the heart pose distinct challenges. Existing masked autoencoder (MAE) approaches typically process images or short clips independently, failing to capture the inherent multi-view structure required for coherent cardiac representation. We introduce Latent Attention Masked Autoencoder (LAMAE), a foundation model architecture tailored to the multi-view nature of medical imaging. LAMAE augments the standard MAE with a latent attention module that enables information exchange across frames and views directly in latent space. This allows the model to aggregate variable-length sequences and distinct views, reconstructing a holistic representation of cardiac function from partial observations. We pretrain LAMAE on MIMIC-IV-ECHO, a large-scale, uncurated dataset reflecting real-world clinical variability. To the best of our knowledge, we present the first results for predicting ICD-10 codes from MIMIC-IV-ECHO videos. Furthermore, we empirically demonstrate that representations learned from adult data transfer effectively to pediatric cohorts despite substantial anatomical differences. These results provide evidence that incorporating structural priors, such as multi-view attention, yields significantly more robust and transferable representations.