US-JEPA: A Joint Embedding Predictive Architecture for Medical Ultrasound

TL;DR

US-JEPA introduces a self-supervised learning framework for ultrasound imaging that leverages a frozen teacher to improve robustness and performance across diverse medical ultrasound tasks.

Contribution

It proposes US-JEPA, a novel latent prediction approach with a static teacher, addressing noise challenges and enabling effective ultrasound representation learning.

Findings

US-JEPA achieves competitive or superior classification performance.

It provides the first comprehensive comparison of ultrasound foundation models.

Masked latent prediction offers stable and efficient ultrasound representations.

Abstract

Ultrasound (US) imaging poses unique challenges for representation learning due to its inherently noisy acquisition process. The low signal-to-noise ratio and stochastic speckle patterns hinder standard self-supervised learning methods relying on a pixel-level reconstruction objective. Joint-Embedding Predictive Architectures (JEPAs) address this drawback by predicting masked latent representations rather than raw pixels. However, standard approaches depend on hyperparameter-brittle and computationally expensive online teachers updated via exponential moving average. We propose US-JEPA, a self-supervised framework that adopts the Static-teacher Asymmetric Latent Training (SALT) objective. By using a frozen, domain-specific teacher to provide stable latent targets, US-JEPA decouples student-teacher optimization and pushes the student to expand upon the semantic priors of the teacher. In addition, we provide the first rigorous comparison of all publicly available state-of-the-art ultrasound foundation models on UltraBench, a public dataset benchmark spanning multiple organs and pathological conditions. Under linear probing for diverse classification tasks, US-JEPA achieves performance competitive with or superior to domain-specific and universal vision foundation model baselines. Our results demonstrate that masked latent prediction provides a stable and efficient path toward robust ultrasound representations.