Med-PU: Point Cloud Upsampling for High-Fidelity 3D Medical Shape Reconstruction

TL;DR

Med-PU is a novel framework that combines medical image segmentation with deep learning-based point cloud upsampling to produce high-fidelity, anatomically accurate 3D reconstructions of pelvic shapes from sparse data.

Contribution

It introduces a knowledge-driven, anatomy-agnostic approach that learns implicit anatomical priors directly from large-scale 3D shape data for improved medical shape reconstruction.

Findings

Consistently improves surface quality and fidelity over state-of-the-art methods.

Reduces artifacts and enhances robustness across different input densities.

Applicable to various skeletal regions and organs beyond the pelvis.

Abstract

High-fidelity 3D anatomical reconstruction is a prerequisite for downstream clinical tasks such as preoperative planning, radiotherapy target delineation, and orthopedic implant design. We present Med-PU, a knowledge-driven framework that integrates volumetric medical image segmentation with point cloud upsampling for accurate pelvic shape reconstruction. Unlike landmark- or PCA-based statistical shape models, Med-PU learns an implicit anatomical prior directly from large-scale 3D shape data, enabling dense completion and refinement from sparse segmentation-derived point sets. The pipeline couples SAM-Med3D-based voxel segmentation, point extraction, deep upsampling, and surface reconstruction, yielding smooth and topologically consistent meshes. We evaluate Med-PU on pelvic CT datasets (MedShapePelvic for training and Pelvic1k for validation), benchmarking against state-of-the-art upsampling methods using comprehensive geometry and surface metrics. Med-PU consistently improves surface quality and anatomical fidelity while reducing artifacts, demonstrating robustness across input densities. Although validated on the pelvis, the approach is anatomy-agnostic and applicable to other skeletal regions and organs. These results suggest Med-PU as a practical, generalizable tool to bridge segmentation outputs and clinically usable 3D models.