A Deep-Learning Approach For Direct Whole-Heart Mesh Reconstruction

TL;DR

This paper introduces a novel deep-learning method that directly reconstructs whole heart surface meshes from volumetric medical images, improving accuracy, anatomical consistency, and temporal coherence over previous segmentation-based approaches.

Contribution

It presents a graph convolutional neural network that deforms a template mesh to accurately reconstruct multiple cardiac structures from CT and MR images, including 4D heart motion.

Findings

Achieved equal or better accuracy than prior methods on CT and MR data.

Generated high-resolution geometries from lower resolution inputs.

Produced temporally consistent heart surface meshes for dynamic imaging.

Abstract

Automated construction of surface geometries of cardiac structures from volumetric medical images is important for a number of clinical applications. While deep-learning-based approaches have demonstrated promising reconstruction precision, these approaches have mostly focused on voxel-wise segmentation followed by surface reconstruction and post-processing techniques. However, such approaches suffer from a number of limitations including disconnected regions or incorrect surface topology due to erroneous segmentation and stair-case artifacts due to limited segmentation resolution. We propose a novel deep-learning-based approach that directly predicts whole heart surface meshes from volumetric CT and MR image data. Our approach leverages a graph convolutional neural network to predict deformation on mesh vertices from a pre-defined mesh template to reconstruct multiple anatomical structures in a 3D image volume. Our method demonstrated promising performance of generating whole heart reconstructions with as good or better accuracy than prior deep-learning-based methods on both CT and MR data. Furthermore, by deforming a template mesh, our method can generate whole heart geometries with better anatomical consistency and produce high-resolution geometries from lower resolution input image data. Our method was also able to produce temporally consistent surface mesh predictions for heart motion from CT or MR cine sequences, and therefore can potentially be applied for efficiently constructing 4D whole heart dynamics. Our code and pre-trained networks are available at https://github.com/fkong7/MeshDeformNet