From Pixels to Polygons: A Survey of Deep Learning Approaches for Medical Image-to-Mesh Reconstruction

TL;DR

This survey reviews deep learning methods for converting medical images into 3D mesh models, categorizing approaches, evaluating their performance, and discussing future research challenges and directions.

Contribution

It provides a comprehensive categorization and analysis of existing deep learning techniques for medical image-to-mesh reconstruction, including datasets, metrics, and future challenges.

Findings

Deep learning approaches are categorized into four main types.

Quantitative evaluation across various anatomical applications.

Identification of key challenges like topological correctness and multi-modality integration.

Abstract

Deep learning-based medical image-to-mesh reconstruction has rapidly evolved, enabling the transformation of medical imaging data into three-dimensional mesh models that are critical in computational medicine and in silico trials for advancing our understanding of disease mechanisms, and diagnostic and therapeutic techniques in modern medicine. This survey systematically categorizes existing approaches into four main categories: template models, statistical models, generative models, and implicit models. Each category is analysed in detail, examining their methodological foundations, strengths, limitations, and applicability to different anatomical structures and imaging modalities. We provide an extensive evaluation of these methods across various anatomical applications, from cardiac imaging to neurological studies, supported by quantitative comparisons using standard metrics. Additionally, we compile and analyze major public datasets available for medical mesh reconstruction tasks and discuss commonly used evaluation metrics and loss functions. The survey identifies current challenges in the field, including requirements for topological correctness, geometric accuracy, and multi-modality integration. Finally, we present promising future research directions in this domain. This systematic review aims to serve as a comprehensive reference for researchers and practitioners in medical image analysis and computational medicine.