Learning Ultrasound Rendering from Cross-Sectional Model Slices for Simulated Training

TL;DR

This paper introduces a method to generate high-quality ultrasound images for training by learning from offline simulated slices, enabling real-time rendering without computational delays during interactive use.

Contribution

It proposes a novel generative adversarial framework that translates cross-sectional model slices into realistic ultrasound images, improving quality without increasing network complexity.

Findings

Achieves comparable or better image quality than existing methods using only tissue maps.

Demonstrates significant improvements through ablation studies and local histogram error metrics.

Enables real-time ultrasound image rendering for training applications.

Abstract

Purpose. Given the high level of expertise required for navigation and interpretation of ultrasound images, computational simulations can facilitate the training of such skills in virtual reality. With ray-tracing based simulations, realistic ultrasound images can be generated. However, due to computational constraints for interactivity, image quality typically needs to be compromised. Methods. We propose herein to bypass any rendering and simulation process at interactive time, by conducting such simulations during a non-time-critical offline stage and then learning image translation from cross-sectional model slices to such simulated frames. We use a generative adversarial framework with a dedicated generator architecture and input feeding scheme, which both substantially improve image quality without increase in network parameters. Integral attenuation maps derived from cross-sectional model slices, texture-friendly strided convolutions, providing stochastic noise and input maps to intermediate layers in order to preserve locality are all shown herein to greatly facilitate such translation task. Results. Given several quality metrics, the proposed method with only tissue maps as input is shown to provide comparable or superior results to a state-of-the-art that uses additional images of low-quality ultrasound renderings. An extensive ablation study shows the need and benefits from the individual contributions utilized in this work, based on qualitative examples and quantitative ultrasound similarity metrics. To that end, a local histogram statistics based error metric is proposed and demonstrated for visualization of local dissimilarities between ultrasound images.