Anatomy Completor: A Multi-class Completion Framework for 3D Anatomy Reconstruction

TL;DR

This paper presents a novel multi-class completion framework using a 3D denoising auto-encoder to reconstruct missing anatomies in medical imaging data, enhancing applications like bio-printing and forensic imaging.

Contribution

It introduces a multi-class 3D anatomy completion method based on a denoising auto-encoder with a new loss aggregation scheme for improved reconstruction accuracy.

Findings

Effective reconstruction of missing anatomies in CT data.

Handles multiple missing anatomies with varying levels of incompleteness.

Code and models are publicly available for reproducibility.

Abstract

In this paper, we introduce a completion framework to reconstruct the geometric shapes of various anatomies, including organs, vessels and muscles. Our work targets a scenario where one or multiple anatomies are missing in the imaging data due to surgical, pathological or traumatic factors, or simply because these anatomies are not covered by image acquisition. Automatic reconstruction of the missing anatomies benefits many applications, such as organ 3D bio-printing, whole-body segmentation, animation realism, paleoradiology and forensic imaging. We propose two paradigms based on a 3D denoising auto-encoder (DAE) to solve the anatomy reconstruction problem: (i) the DAE learns a many-to-one mapping between incomplete and complete instances; (ii) the DAE learns directly a one-to-one residual mapping between the incomplete instances and the target anatomies. We apply a loss aggregation scheme that enables the DAE to learn the many-to-one mapping more effectively and further enhances the learning of the residual mapping. On top of this, we extend the DAE to a multiclass completor by assigning a unique label to each anatomy involved. We evaluate our method using a CT dataset with whole-body segmentations. Results show that our method produces reasonable anatomy reconstructions given instances with different levels of incompleteness (i.e., one or multiple random anatomies are missing). Codes and pretrained models are publicly available at https://github.com/Jianningli/medshapenet-feedback/ tree/main/anatomy-completor