Three-Dimensional Anatomical Data Generation Based on Artificial Neural Networks

TL;DR

This paper introduces a novel workflow using physical organ models and GANs to generate 3D anatomical data, overcoming challenges in data collection for soft tissues like the prostate.

Contribution

It presents a new method combining physical models and neural networks to automate 3D data generation for medical applications.

Findings

Neural network segmentation outperforms traditional methods in IoU.

Generated 3D models are useful for downstream machine learning tasks.

Workflow successfully simulates endoscopic procedures on artificial organs.

Abstract

Surgical planning and training based on machine learning requires a large amount of 3D anatomical models reconstructed from medical imaging, which is currently one of the major bottlenecks. Obtaining these data from real patients and during surgery is very demanding, if even possible, due to legal, ethical, and technical challenges. It is especially difficult for soft tissue organs with poor imaging contrast, such as the prostate. To overcome these challenges, we present a novel workflow for automated 3D anatomical data generation using data obtained from physical organ models. We additionally use a 3D Generative Adversarial Network (GAN) to obtain a manifold of 3D models useful for other downstream machine learning tasks that rely on 3D data. We demonstrate our workflow using an artificial prostate model made of biomimetic hydrogels with imaging contrast in multiple zones. This is used to physically simulate endoscopic surgery. For evaluation and 3D data generation, we place it into a customized ultrasound scanner that records the prostate before and after the procedure. A neural network is trained to segment the recorded ultrasound images, which outperforms conventional, non-learning-based computer vision techniques in terms of intersection over union (IoU). Based on the segmentations, a 3D mesh model is reconstructed, and performance feedback is provided.