Deep Motion Network for Freehand 3D Ultrasound Reconstruction

TL;DR

This paper introduces MoNet, a deep learning approach that combines ultrasound images and IMU data to improve freehand 3D ultrasound reconstruction accuracy by addressing elevational displacement and drift issues.

Contribution

The paper presents a novel deep motion network that incorporates IMU acceleration and a self-supervised strategy for enhanced 3D ultrasound reconstruction.

Findings

Achieves superior reconstruction accuracy over existing methods.

Effectively estimates elevational displacements using IMU acceleration.

Reduces drift errors through adaptive self-supervised optimization.

Abstract

Freehand 3D ultrasound (US) has important clinical value due to its low cost and unrestricted field of view. Recently deep learning algorithms have removed its dependence on bulky and expensive external positioning devices. However, improving reconstruction accuracy is still hampered by difficult elevational displacement estimation and large cumulative drift. In this context, we propose a novel deep motion network (MoNet) that integrates images and a lightweight sensor known as the inertial measurement unit (IMU) from a velocity perspective to alleviate the obstacles mentioned above. Our contribution is two-fold. First, we introduce IMU acceleration for the first time to estimate elevational displacements outside the plane. We propose a temporal and multi-branch structure to mine the valuable information of low signal-to-noise ratio (SNR) acceleration. Second, we propose a multi-modal online self-supervised strategy that leverages IMU information as weak labels for adaptive optimization to reduce drift errors and further ameliorate the impacts of acceleration noise. Experiments show that our proposed method achieves the superior reconstruction performance, exceeding state-of-the-art methods across the board.