MedTet: An Online Motion Model for 4D Heart Reconstruction

TL;DR

MedTet introduces a flexible online framework for reconstructing 3D cardiac motion from sparse intraoperative data, enabling real-time, anatomically consistent heart motion modeling during surgeries.

Contribution

It presents a novel deformable tetrahedral grid-based method that reconstructs 3D cardiac motion from limited data types in real-time, a significant advance over existing full-volume imaging techniques.

Findings

Accurately reconstructs 3D cardiac motion from sparse data

Generates plausible, anatomically consistent motion during interventions

Works with 2D slices, 1D signals, and full 3D volumes

Abstract

We present a novel approach to reconstruction of 3D cardiac motion from sparse intraoperative data. While existing methods can accurately reconstruct 3D organ geometries from full 3D volumetric imaging, they cannot be used during surgical interventions where usually limited observed data, such as a few 2D frames or 1D signals, is available in real-time. We propose a versatile framework for reconstructing 3D motion from such partial data. It discretizes the 3D space into a deformable tetrahedral grid with signed distance values, providing implicit unlimited resolution while maintaining explicit control over motion dynamics. Given an initial 3D model reconstructed from pre-operative full volumetric data, our system, equipped with an universal observation encoder, can reconstruct coherent 3D cardiac motion from full 3D volumes, a few 2D MRI slices or even 1D signals. Extensive experiments on cardiac intervention scenarios demonstrate our ability to generate plausible and anatomically consistent 3D motion reconstructions from various sparse real-time observations, highlighting its potential for multimodal cardiac imaging. Our code and model will be made available at https://github.com/Scalsol/MedTet.