OnUVS: Online Feature Decoupling Framework for High-Fidelity Ultrasound Video Synthesis

TL;DR

OnUVS is an innovative online framework that synthesizes high-fidelity ultrasound videos by decoupling features, incorporating anatomical information, and constraining motion, thereby aiding medical training and diagnosis.

Contribution

The paper introduces a novel online feature-decoupling framework with weakly-supervised anatomical keypoint learning and dual-decoder architecture for realistic ultrasound video synthesis.

Findings

Synthesizes ultrasound videos with high fidelity.

Improves anatomical and motion preservation in generated videos.

Enhances sharpness and detail through a multi-feature discriminator.

Abstract

Ultrasound (US) imaging is indispensable in clinical practice. To diagnose certain diseases, sonographers must observe corresponding dynamic anatomic structures to gather comprehensive information. However, the limited availability of specific US video cases causes teaching difficulties in identifying corresponding diseases, which potentially impacts the detection rate of such cases. The synthesis of US videos may represent a promising solution to this issue. Nevertheless, it is challenging to accurately animate the intricate motion of dynamic anatomic structures while preserving image fidelity. To address this, we present a novel online feature-decoupling framework called OnUVS for high-fidelity US video synthesis. Our highlights can be summarized by four aspects. First, we introduced anatomic information into keypoint learning through a weakly-supervised training strategy, resulting in improved preservation of anatomical integrity and motion while minimizing the labeling burden. Second, to better preserve the integrity and textural information of US images, we implemented a dual-decoder that decouples the content and textural features in the generator. Third, we adopted a multiple-feature discriminator to extract a comprehensive range of visual cues, thereby enhancing the sharpness and fine details of the generated videos. Fourth, we constrained the motion trajectories of keypoints during online learning to enhance the fluidity of generated videos. Our validation and user studies on in-house echocardiographic and pelvic floor US videos showed that OnUVS synthesizes US videos with high fidelity.