LOTUS: Learning to Optimize Task-based US representations

TL;DR

This paper introduces a novel method that uses simulated ultrasound images generated from CT data and a differentiable simulator to improve organ segmentation in ultrasound images, reducing reliance on extensive labeled data.

Contribution

It presents a physics-based, differentiable ultrasound simulation framework combined with an image adaptation network for end-to-end training to enhance US segmentation accuracy.

Findings

Promising quantitative segmentation results on aorta and vessel tasks.

Effective image synthesis and adaptation between real and simulated US images.

Potential to reduce the need for large labeled ultrasound datasets.

Abstract

Anatomical segmentation of organs in ultrasound images is essential to many clinical applications, particularly for diagnosis and monitoring. Existing deep neural networks require a large amount of labeled data for training in order to achieve clinically acceptable performance. Yet, in ultrasound, due to characteristic properties such as speckle and clutter, it is challenging to obtain accurate segmentation boundaries, and precise pixel-wise labeling of images is highly dependent on the expertise of physicians. In contrast, CT scans have higher resolution and improved contrast, easing organ identification. In this paper, we propose a novel approach for learning to optimize task-based ultra-sound image representations. Given annotated CT segmentation maps as a simulation medium, we model acoustic propagation through tissue via ray-casting to generate ultrasound training data. Our ultrasound simulator is fully differentiable and learns to optimize the parameters for generating physics-based ultrasound images guided by the downstream segmentation task. In addition, we train an image adaptation network between real and simulated images to achieve simultaneous image synthesis and automatic segmentation on US images in an end-to-end training setting. The proposed method is evaluated on aorta and vessel segmentation tasks and shows promising quantitative results. Furthermore, we also conduct qualitative results of optimized image representations on other organs.