MoNetV2: Enhanced Motion Network for Freehand 3D Ultrasound Reconstruction

TL;DR

MoNetV2 introduces an advanced deep learning framework for freehand 3D ultrasound reconstruction, significantly improving accuracy and robustness across diverse scanning conditions by integrating multi-level consistency and self-supervision.

Contribution

The paper presents MoNetV2, a novel motion network that fuses image and motion data, employs multi-level consistency constraints, and uses self-supervised learning to enhance freehand 3D ultrasound reconstruction.

Findings

MoNetV2 outperforms existing methods in reconstruction quality.

It demonstrates superior generalizability across multiple datasets.

The approach effectively reduces cumulative drift and errors.

Abstract

Three-dimensional (3D) ultrasound (US) aims to provide sonographers with the spatial relationships of anatomical structures, playing a crucial role in clinical diagnosis. Recently, deep-learning-based freehand 3D US has made significant advancements. It reconstructs volumes by estimating transformations between images without external tracking. However, image-only reconstruction poses difficulties in reducing cumulative drift and further improving reconstruction accuracy, particularly in scenarios involving complex motion trajectories. In this context, we propose an enhanced motion network (MoNetV2) to enhance the accuracy and generalizability of reconstruction under diverse scanning velocities and tactics. First, we propose a sensor-based temporal and multi-branch structure that fuses image and motion information from a velocity perspective to improve image-only reconstruction accuracy. Second, we devise an online multi-level consistency constraint that exploits the inherent consistency of scans to handle various scanning velocities and tactics. This constraint exploits both scan-level velocity consistency, path-level appearance consistency, and patch-level motion consistency to supervise inter-frame transformation estimation. Third, we distill an online multi-modal self-supervised strategy that leverages the correlation between network estimation and motion information to further reduce cumulative errors. Extensive experiments clearly demonstrate that MoNetV2 surpasses existing methods in both reconstruction quality and generalizability performance across three large datasets.