3D Surface-to-Structure Translation using Deep Convolutional Networks

TL;DR

This paper presents a deep learning system that estimates internal body structures from external 3D surface models, potentially reducing the need for invasive imaging procedures.

Contribution

It introduces a novel deep convolutional neural network approach trained on CT data to predict internal anatomy from surface scans.

Findings

Successful estimation of internal structures from surface models

Use of CT datasets for training the neural network

Potential for non-invasive disease localization

Abstract

Our demonstration shows a system that estimates internal body structures from 3D surface models using deep convolutional neural networks trained on CT (computed tomography) images of the human body. To take pictures of structures inside the body, we need to use a CT scanner or an MRI (Magnetic Resonance Imaging) scanner. However, assuming that the mutual information between outer shape of the body and its inner structure is not zero, we can obtain an approximate internal structure from a 3D surface model based on MRI and CT image database. This suggests that we could know where and what kind of disease a person is likely to have in his/her body simply by 3D scanning surface of the body. As a first prototype, we developed a system for estimating internal body structures from surface models based on Visible Human Project DICOM CT Datasets from the University of Iowa Magnetic Resonance Research Facility.