DiffUS: Differentiable Ultrasound Rendering from Volumetric Imaging

TL;DR

DiffUS is a novel differentiable ultrasound rendering framework that synthesizes realistic intraoperative ultrasound images from volumetric MRI data, enabling improved surgical guidance and image registration.

Contribution

It introduces a physics-based, differentiable ultrasound renderer that converts MRI to ultrasound images using machine learning and ray tracing, facilitating gradient-based optimization.

Findings

Accurately synthesizes ultrasound images from MRI data.

Enables gradient-based registration and reconstruction tasks.

Demonstrates effectiveness on brain imaging datasets.

Abstract

Intraoperative ultrasound imaging provides real-time guidance during numerous surgical procedures, but its interpretation is complicated by noise, artifacts, and poor alignment with high-resolution preoperative MRI/CT scans. To bridge the gap between reoperative planning and intraoperative guidance, we present DiffUS, a physics-based, differentiable ultrasound renderer that synthesizes realistic B-mode images from volumetric imaging. DiffUS first converts MRI 3D scans into acoustic impedance volumes using a machine learning approach. Next, we simulate ultrasound beam propagation using ray tracing with coupled reflection-transmission equations. DiffUS formulates wave propagation as a sparse linear system that captures multiple internal reflections. Finally, we reconstruct B-mode images via depth-resolved echo extraction across fan-shaped acquisition geometry, incorporating realistic artifacts including speckle noise and depth-dependent degradation. DiffUS is entirely implemented as differentiable tensor operations in PyTorch, enabling gradient-based optimization for downstream applications such as slice-to-volume registration and volumetric reconstruction. Evaluation on the ReMIND dataset demonstrates DiffUS's ability to generate anatomically accurate ultrasound images from brain MRI data.